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ABSTRACT 


This  study  is  a  practical  example  of  economic  analysis  of  information  systems  and 
of  the  software  cost  estimation  problem  as  applied  to  software  development  in  the 
Department  of  Defense.  Economic  analysis  methods  and  the  difficulty  of  software  cost 
estimation  are  demonstrated  using  the  proposed  redesign  of  the  Reserve  Financial 
Management  System  (RESFMS),  an  information  system  operated  by  the  U.S.  Naval 
Reserve.  The  mandate  for  economic  analysis  in  the  Department  of  Defense  and 
procedures  applicable  to  information  systems  are  discussed.  Two  alternatives  are 
analyzed:  the  status  quo  and  a  redesign  proposed  by  Commander  Naval  Reserve  Force 
(COMNAVRESFOR).  Costs  to  be  considered  for  each  alternative  are  described.  Since 
the  major  cost  of  the  redesign  will  be  software  development,  the  problem  of  software 
development  cost  estimation  is  discussed.  An  estimate  of  software  development  cost  is 
produced.  This  estimate  and  other  identified  costs  are  used  to  calculate  present  value  of 
savings,  savings/investment  ratio,  and  discounted  payback  period  for  the  redesign 
alternative  as  compared  to  the  status  quo.  Risk  analysis,  using  a  monte  carlo  simulation, 
is  then  performed  to  determine  the  range  of  possible  outcome  values  and  probabilities  for 
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I.  INTRODUCTION 


A.  BACKGROUND 

Intuitively  the  proposal  seemed  economically  beneficial  to  the  Naval  Reserve:  take 
an  information  system  costing  over  three  million  dollars  per  year  to  operate  on  a 
mainframe  computer  and  move  the  system  to  a  network  of  minicomputers  costing  a  few 
hundred  thousand  dollars  to  purchase  and  the  same  each  year  to  operate.  The  savings 
should  be  substantial  and  system  support  might  even  improve,  since  the  operators  of  the 
mainframe  had  not  been  very  responsive  to  user  requests  lately.  This  was  the  proposal 
presented  to  me  when  Commander  Naval  Reserve  Force  (COMNAVRESFOR  or 
CNRF)1  staff  members  asked,  in  July  1991,  for  an  economic  analysis  of  the  redesign 
alternatives  for  the  Reserve  Financial  Management  System  (RESFMS).  As  it  turns  out, 
more  than  a  year  later,  the  outcome  of  the  analysis  produced  results  similar  to  those 
which  intuition  suggested.  However,  the  process  of  evaluation  uncovered  questions  and 
raised  issues  not  originally  considered  which  are  of  great  import  to  the  success  of  any 
redesign  effort.  Also,  no  matter  how  strongly  managers  are  convinced  that  an 
information  system  development  project  would  be  beneficial,  if  that  judgement  is  not 
based  in  objective  analysis,  funds  for  development  will  not  be  approved  in  the 

'Members  of  the  Naval  Reserve  generally  refer  to  Commander  Naval  Reserve  Force 
as  COMNAVRESFOR.  For  purposes  of  brevity  in  correspondence  or  references  it  is 
sometimes  shortened  to  the  four  letters  CNRF.  This  usage  will  be  preserved  such  that 
the  body  of  the  text  will  use  COMNAVRESFOR,  and  tables,  graphs,  and  references  will 
generally  use  CNRF. 
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Department  of  Defense  (DoD).  Therefore,  a  formal  analysis  of  alternatives  was  not  only 
instructive  to  the  planning  process,  but  required  by  DoD  policy. 

B.  OBJECTIVES 

The  primary  objective  of  the  study  was  to  ascertain  the  economic  benefits,  if  any, 
to  the  Naval  Reserve  of  redesigning  RESFMS  to  operate  in  a  hardware  environment 
other  than  the  mainframe  on  which  it  runs  at  present.  This  analysis  was  to  be  done  in 
a  manner  consistent  with  current  DoD  directives  and  following  established  economic 
analysis  principles. 

The  target  hardware  configuration  for  the  redesign  was  determined  by 
COMNAVRESFOR  personnel  and  presented  to  me  for  use  in  the  economic  analysis. 
Yet,  although  the  type  of  computer  to  be  used  was  determined,  the  number  of 
minicomputers  required  to  run  RESFMS  was  not  determined  prior  to  my  evaluation. 
Therefore,  one  of  the  first  objectives  of  this  study  was  to  validate  the  adequacy  of  the 
computer  chosen  to  effectively  operate  RESFMS,  and  to  determine  the  number  of 
computers  required  for  this  purpose. 

The  next  objective  of  the  study  was  to  determine  all  current  system  costs  and 
benefits  and  to  estimate  alternative  system  costs  and  benefits.  Determination  of  current 
system  costs  was  fairly  straightforward.  However,  estimation  of  alternative  system  costs 
turned  out  to  be  the  most  difficult  part  of  this  entire  study.  The  determination  of 
hardware  costs  was  relatively  simple.  The  estimation  of  the  cost  of  developing  new 
system  software,  on  the  other  hand,  was  a  daunting  problem  which  caused  further  study 
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and  analysis  of  the  problem  of  software  cost  estimation  in  general,  and  of  the  problem 
in  the  Department  of  Defense,  in  particular. 

The  final  objective  of  this  study  was  to  analyze  the  costs  and  benefits  identified  in 
such  a  way  that  decision  makers  could  use  the  analysis  as  a  basis  for  program 
development  approval  or  disapproval.  In  keeping  with  this  objective,  an  analysis  has 
been  made  using  several  economic  analysis  tools  and  risk  analysis  procedures  applied  to 
the  results.  Consequently,  the  final  outcome  of  the  study  provides  not  single  values,  but 
ranges  of  values  and  probabilities  of  outcomes  for  decision  maker  evaluation. 

C.  SCOPE,  LIMITATIONS,  AND  ASSUMPTIONS 

This  study  is  limited  to  the  alternatives  and  data  provided  by  COMNAVRESFOR 
for  analysis.  General  principles  and  procedures  examined,  such  as  economic  analysis, 
software  development  cost  estimation,  and  risk  analysis,  will  be  discussed  as  they  relate 
to  information  systems  development  in  the  Department  of  Defense.  Assumptions  related 
to  specific  items  of  the  analysis  will  be  discussed  when  those  items  are  described  and 
evaluated. 

D.  ORGANIZATION  OF  STUDY 

One  reason  why  an  economic  analysis  of  redesign  alternatives  for  RESFMS  is 
being  performed  is  because  such  an  analysis  is  required  within  DoD.  To  understand 
what  is  required  and  why,  it  is  helpful  to  examine  the  background  of  economic  analysis 
in  general  as  well  as  the  history  and  current  requirements  for  economic  analysis  in  DoD. 
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These  issues,  along  with  a  description  of  assumptions  for  this  study  that  result  from 
current  DoD  directives,  will  be  discussed  first. 

In  order  to  understand  the  rationale  for  a  proposed  redesign  of  RESFMS,  the 
history  of  its  development  and  evolution  should  be  understood.  Knowledge  of  the  current 
configuration  is  essential  to  understanding  current  operating  costs  and  benefits;  and  the 
target  configuration  and  rationale  used  to  determine  it  are  useful  in  understanding  the 
potential  costs  and  benefits  of  the  redesign  alternative.  Therefore,  a  discussion  of  these 
aspects  of  RESFMS  will  be  next. 

Since  the  process  of  economic  analysis  is  central  to  this  study,  and  the  selection  of 
economic  analysis  tools  critical  to  the  results  obtained,  the  structure  of  economic 
analysis,  definitions  of  major  terms,  and  description  of  appropriate  tools  will  be 
examined.  The  economic  analysis  tools  selected  for  this  study  will  be  listed  and  briefly 
explained. 

The  problem  of  estimating  the  cost  of  software  development  for  RESFMS  was  the 
most  significant  problem  encountered  in  this  study.  Therefore,  a  chapter  will  be  devoted 
to  an  explanation  of  the  nature  of  the  software  cost  estimation  problem  in  general,  some 
of  the  methods  and  tools  available  to  produce  estimates,  and  the  applicability  of  this 
problem  to  information  systems  development  in  DoD. 

Next,  a  detailed  description  of  costs  chosen  for  this  analysis  is  presented.  Special 
attention  is  given  to  the  source  data,  reasoning,  assumptions  and  methods  used  to 
determine  software  development  cost  estimates  in  the  case  of  RESFMS. 
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Once  the  tools  have  been  chosen  and  all  costs  identified  and  quantified,  the  analysis 
is  performed.  The  first  computations  of  economic  analysis  were  done  using  actual  costs, 
if  known,  and  expected  values  if  cost  was  uncertain.  Then,  risk  analysis  is  performed 
using  the  full  range  of  possible  values  for  uncertain  cost  estimates.  The  results  of  risk 
analysis  show  a  range  of  possible  outcomes  and  the  probability  of  obtaining  those  values. 

Finally,  a  recommendation  will  be  made,  based  on  the  results  of  economic  analysis, 
as  to  what  alternative  will  have  the  highest  probability  of  positive  financial  benefit  for 
the  Naval  Reserve. 
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n.  BACKGROUND 


A.  ECONOMIC  ANALYSIS 

The  efficiency  of  capital  expenditures  has  been  acknowledged  as  contributing  to  the 
success  of  both  private  business  firms  and  government.  "Not  only  do  capital  expenditure 
decisions  generally  represent  significant  sums  of  money,  they  also  affect  future  years  and 
are  often  irreversible"  (Abdelsamad,  1970).  Because  capital  expenditure  decisions  are 
important,  a  number  of  techniques  have  been  developed  to  aid  organizations  in  the 
decision  process.  These  techniques  fall  under  the  broad  category  of  economic  analysis 
and  a  body  of  literature  has  emerged  that  describes  the  techniques  and  their  use. 

At  the  heart  of  economic  analysis  is  the  concept  of  the  time  value  of  money.  Levy 
and  Samat  (1982)  as  well  as  Garrison  (1988)  point  out  that  factors  such  as  interest  rates 
and  inflation  cause  the  value  of  a  dollar  received  today  to  be  greater  than  a  dollar 
received  in  the  future.  The  same  is  true  of  costs.  Expenditures  today  cost  more,  in  real 
terms,  than  the  same  amount  of  expenditures  at  some  future  date.  Therefore,  our 
methods  of  analysis  must  account  for  the  differing  value  of  both  costs  and  benefits  in 
relation  to  time. 

It  is  important  to  understand  that  economic  analysis  does  not,  by  itself,  produce 

decisions.  The  results  of  economic  analysis  still  contain  uncertainty. 

...by  its  basic  nature,  an  economic  evaluation  requires  the  estimation  of  many 
variables,  usually  over  a  considerable  period  of  time.  Consequently,  there  is 
always  an  element  of  risk  in  any  economic  evaluation.  (Stevens,  1979) 
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While,  economic  analyses  provide  guidelines  for  making  decisions,  some  person  or  group 
must  ultimately  accept  the  risk  and  make  the  decisions. 


B.  INFORMATION  SYSTEM  EXPENDITURES 

In  the  last  three  decades,  rapid  advancement  in  information  systems  (IT)  technology 

has  resulted  in  major  investment  in  IT  hardware,  software,  and  related  systems  by  both 

large  corporations  and  very  small  firms  (Cash  and  others,  1988).  An  indication  of  the 

scope  of  these  investments  and  investing  trends  can  be  obtained  by  looking  at  capital 

investment  in  high-technology  industries. 

It  is  clear  that  United  States  capital  investment  is  increasingly  turning  towards  high- 
technology  industries.  ...  Capital  expenditures  for  basic  industrial  equipment  have 
been  reduced  from  25%  of  all  capital  spending  in  the  1960s  to  present  levels  of  less 
than  13%  of  all  capital  spending.  Meanwhile,  spending  for  high-tech  equipment 
rose  from  12%  in  the  1960s  to  present  levels  of  more  than  30%  of  all  capital 
spending.  (Strassmann,  1985) 

The  need  to  analyze  these  growing  expenditures  and  informed  decisions  has  caused 
organizations  to  apply  economic  analysis  methods  to  ADPE  acquisition  and  information 
system  development. 


C.  GOVERNMENT  REGULATIONS  CONCERNING  ADPE 

The  Congress  of  the  United  States,  recognizing  the  importance  of  capital 
expenditure  on  ADPE,  has  enacted  legislation  and  given  policy  direction  through  hearings 
and  reports.  In  October  1965  Congress  enacted  Public  Law  89-306,  known  as  the 
Brooks  Act,  establishing  the  basic  policy  for  the  management  of  data  processing 
equipment  in  the  Federal  Government.  Public  Law  99-500,  known  as  the  Paperwork 


7 


Reduction  Reauthorization  Act  of  1986,  expanded  the  scope  of  the  Brooks  Act  "to 

include  telecommunications  resources,  software,  and  computer-related  services  such  as 

computer  service  bureaus  and  contract  programming.”  (GSA  Overview  Guide,  1990) 

The  Brooks  Act  charges  the  Office  of  Management  and  Budget  (OMB)  with 

developing  management  policy  and  providing  fiscal  control  of  ADPE.  In  its  circular 

entitled  "Management  of  Federal  Information  Resources",  the  general  principles  of 

ADPE  acquisition  and  development  are  stated.  One  of  the  principles  cited  states: 

In  order  to  minimize  the  cost  and  maximize  the  usefulness  of  government 
information  activities,  the  expected  public  and  private  benefits  derived  from 
government  information,  insofar  as  they  are  calculable,  should  exceed  the  public 
and  private  costs  of  the  information.  (OMB  Circular  A-130,  1985) 

To  determine  if  benefits  derived  do,  in  fact,  exceed  the  costs,  one  must  use  some  form 

of  economic  analysis.  Procedures  and  policies  for  economic  analysis  of  ADPE  in  the 

government  are  delineated  by  the  General  Services  Administration  (GSA). 

According  to  the  Brooks  Act,  GSA  is  to  "coordinate  and  provide  for  the  economic 

and  efficient  purchase,  lease,  and  maintenance  of  automated  data  processing  equipment 

by  Federal  agencies."  Although  GSA  is  given  exclusive  authority  to  procure  ADPE 

resources,  it  is  also  granted  the  power  to  delegate  authority  for  procurement  to  Federal 

agencies  as  necessary.  (GSA  Overview  Guide,  1990) 

The  primary  document  containing  GSA  regulations  for  automated  data  processing 

equipment  is  the  Federal  Information  Resources  Management  Regulation  (FIRMR).  In 

the  FIRMR,  automated  data  processing  equipment  and  associated  information  systems  are 

referred  to  as  Federal  Information  Processing  (FIP)  resources.  In  its  section  on 
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acquisition,  the  FIRMR  directs  that  an  analysis  of  alternatives  be  made  prior  to  acquiring 
or  developing  any  system  and  that  "in  the  analysis  of  alternatives,  agencies  shall  calculate 
the  total  estimated  cost,  using  the  present  value  of  money,  for  each  feasible  alternative 
unless  the  cost  of  the  acquisition  is  $50,000  or  less."  (FIRMR,  1990)  It  also  directs, 
when  calculating  the  cost  of  each  alternative,  agencies  must  follow  guidance  in  OMB 
Circular  No.  A-94,  "Discount  Rates  to  be  Used  in  Evaluating  Time-Distributed  Costs  and 
Benefits."2  The  FIRMR,  further,  specifically  lists  those  costs  to  be  included  and  those 
to  be  excluded  in  any  analysis. 

Using  its  power  to  delegate,  GSA  has  set  criteria  to  determine  which  acquisitions 
and  development  efforts  must  be  reviewed  and  approved  by  GSA  itself,  and  which  may 
be  approved  by  the  government  agency  acquiring  the  system.  Each  agency,  in  tum,  has 
produced  its  own  set  of  regulations  and  directives  governing  FIP  resource  acquisition 
within  that  agency. 

D.  FIP  RESOURCE  ACQUISITION  AND  MANAGEMENT  IN  DOD 

The  armed  services  are  the  heaviest  user  of  FIP  resources  in  the  U.S.  Government 
(Kellner,  1991),  spending  almost  nine  billion  dollars  on  automatic  data  processing  in 


2OMB  Circular  A-94  specifically  states:  "This  Circular  would  not  apply  to  the 
evaluation  of  decisions  concerning  how  to  select  automatic  data  processing  equipment, 
guidance  for  which  is  OMB  Circular  No.  A-54  and  OMB  Bulletin  No.  60-6."  Yet, 
GSA,  in  the  FIRMR,  §201-20.203-1  (c),  directs:  "Agencies  shall  follow  guidance  in 
OMB  Circular  No.  A-94,  ’Discount  Rates  to  be  Used  in  Evaluating  Time-Distributed 
Costs  and  Benefits,’  when  calculating  the  cost  of  each  alternative."  Since  the  FIRMR 
is  the  most  recent  of  the  documents  (1990  vice  1972),  it  takes  precedence  and  procedures 
in  OMB  Circular  A-94  are  to  be  followed  in  spite  of  the  disclaimer. 
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1990  alone  (HASC,  1989).  To  manage  these  resources,  the  Department  of  Defense 
(DoD)  has  established  its  own  procedures  and  regulations  regarding  FIP  resources.  DoD 
Directive  7920.1,  "Life-Cycle  Management  (LCM)  of  Automated  Information  Systems 
(AISs),"  DoD  Directive  7920.2,  "Automated  Information  System  (AIS)  Life-Cycle 
Management  Review  and  Milestone  Approval  Procedures,"  and  DoD  Directive  5000.1, 
"Defense  Acquisition,"  all  require  that  regulations  found  in  the  FIRMR  be  followed  in 
FIP  resource  acquisition.  As  noted  earlier,  the  FIRMR  requires  an  economic  analysis 
of  alternatives.  Within  DoD,  procedures  to  be  followed  in  this  economic  analysis  are 
found  in  DoD  Directive  7041.3,  "Economic  Analysis  and  Program  Evaluation  for 
Resource  Management."  Individual  services,  including  the  Department  of  the  Navy 
(DoN)  have  written  their  own  directives,  following  OMB,  GSA,  and  DoD  guidance, 
which  further  spell  out  procedures  to  be  followed  in  each  service.  The  general  principles 
of  economic  analysis  found  in  methods  used  in  the  private  sector  are  found  in  these 
directives  as  well.  Thus,  careful  consideration  is  to  be  given  to  the  time  value  of  money 
and  the  effects  of  interest  rates  and  inflation. 

In  spite  of  this  plethora  of  direction  concerning  automatic  data  processing  systems, 
DoD  has  experienced  significant  problems  analyzing,  acquiring,  developing  and 
managing  these  systems.  A  series  of  reports  from  both  houses  of  Congress  between  1988 
and  1990  documented  these  problems.  Responding  to  GAO  reports  of  mismanagement 
of  ADPE  in  DoD,  the  House  Armed  Services  Committee  (HASC)  recommended,  in  July 
1989,  that  the  services’  automatic  data  processing  request  be  reduced  by  $165.5  million. 
The  HASC  also  stipulated  a  requirement  that  DoD  develop  a  plan  of  action  by  February 
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1,  1990  on  how  to  resolve  the  identified  problems.  The  implication  was  that  further 
reductions  would  result  if  the  deadline  were  not  met. 

E.  DOD  CORPORATE  INFORMATION  MANAGEMENT  INITIATIVE 

In  response  to  the  above  HASC  requirements,  Deputy  Secretary  of  Defense  Donald 

Atwood  announced,  in  October  1989,  a  Corporate  Information  Management  (CIM) 

initiative.  This  initiative  involves  radical  changes  in  the  way  information  resources  are 

managed  in  DoD.  A  thorough  discussion  of  the  tenets  and  implications  of  CIM  is 

beyond  the  scope  of  this  thesis.  However,  it  is  important  to  understand  the  basic  premise 

of  CIM  and  its  effects  on  automatic  data  processing  system  development. 

Central  to  the  philosophy  and  method  of  CIM  is  the  concept  that  "it  is  not  about 

technology;  it  is  about  business  processes  and  managing  information."  (Brewin,  1991) 

The  theory  is  that  businesses  "gain  strategic  advantage  by  changing  the  way  they  work, 

not  by  automating  old  or  inefficient  methods."  (Brewin,  1991)  Thus,  CIM  seeks  to  have 

all  DoD  agencies  analyze  their  basic  business  processes.  Once  a  business  process  is 

understood,  it  can  be  redesigned  with  the  goal  of  achieving  the  greatest  efficiency 

possible  in  every  business  activity.  Then,  and  only  then,  is  information  technology 

considered  as  a  means  of  implementing  this  process. 

By  supporting  functional  managers  in  streamlining  business  methods,  DoD’s 
corporate  information  management  initiative  will  aid  the  Department  in  achieving 
the  aggressive  savings  targets  established  by  the  Defense  Management  Report.  To 
achieve  the  highest  savings,  CIM  investments  must  be  based  on  a  functional 
economic  analysis  of  business  activities  or  operations.  (DDI  Memo,  1991) 
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The  functional  economic  analysis  now  required  for  all  CIM,  and  thus  ADPE, 
investment  decisions  has  been  spelled  out  in  memoranda  and  training  presentations  to 
DoD  management  personnel.  The  functional  economic  analysis,  also  called  a  Business 
Case,  follows  and  amplifies  upon  policy  and  procedures  contained  in  DoD  Directive 
7041.3  described  above.  One  part  of  a  Business  Case  is  an  analysis  of  alternatives  for 
information  systems  to  support  a  redesigned  business  process.  This  analysis  of 
alternatives  is  to  follow  the  guidelines  set  forth  in  the  FIRMR,  and  in  OMB,  DoD,  and 
Department  of  the  Navy  (DoN)  directives  except  as  amended  by  CIM.  The  chief  effects 
of  CIM  on  this  part  of  the  analysis  are  an  increased  emphasis  on  the  financial  impacts 
of  risk,  and  the  requirement  to  express  potential  benefits  in  cash  terms.  Specific 
procedures  will  be  discussed  in  Chapter  IV. 

The  directives  and  guidelines  discussed  so  far,  all  apply  to  all  agencies  within  DoD. 
The  Naval  Reserve,  as  part  of  the  Department  of  the  Navy  (DoN),  is  a  DoD  agency. 
Commander  Naval  Reserve  Force  (COMNAVRESFOR)  is  responsible  for  operating  and 
developing  several  information  systems.  One  of  those  systems  is  the  Reserve  Financial 
Management  Systems  (RESFMS),  which  has  been  proposed  as  a  candidate  for  redesign. 
Project  approval  and  allocation  of  funds  for  the  redesign  of  RESFMS  will  be  contingent 
upon  the  results  of  a  Business  Case  presented  by  COMNAVRESFOR  to  higher  approval 
authority. 


12 


F.  RESFMS  BUSINESS  CASE  ASSUMPTIONS 


The  purpose  of  this  thesis  is  to  perform  an  economic  analysis  of  the  alternatives  for 
the  Reserve  Financial  Management  System  (RESFMS).  Such  an  economic  analysis  will 
be  part  of  the  Business  Case  presented  by  COMNAVRESFOR  to  gain  approval  for 
proceeding  with  the  redesign  effort.  The  other  usual  component  of  a  Business  Case  is 
an  analysis  of  the  business  processes  of  the  agency  making  the  proposal.  An  analysis  and 
redesign  of  the  business  process  of  the  Naval  Reserve  is  beyond  the  scope  of  this  thesis. 
I  assume  that  such  an  analysis,  if  required,  has  already  been  done  and  that  the  functional 
requirements  of  RESFMS,  generated  by  COMNAVRESFOR,  support  efficient  business 
processes  of  the  Naval  Reserve.  I  further  assume  that  a  decision  has  already  been  made 
that  an  information  system,  in  the  form  of  RESFMS,  provides  the  best  means  to  perform 
the  functional  requirements  given. 

This  analysis,  then,  will  meet  the  requirements  for  an  analysis  of  alternatives  for 
an  information  system  that  may  be  a  part  of  a  Business  Case  to  be  produced  by  the  Naval 
Reserve  and  used  as  a  decision  tool  when  considering  future  budget  and  development 
plans. 
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ffl.  RESERVE  FINANCIAL  MANAGEMENT  SYSTEM  (RESFMS) 


A.  DESCRIPTION  OF  RESFMS 

The  Reserve  Financial  Management  System  (RESFMS)  is  an  information  system 
used  by  the  Naval  Reserve  to  manage  the  Reserve  Personnel  Navy  (RPN)  appropriation 
account,  to  issue  active  duty  orders  to  reservists,  and  to  arrange  for  travel  for  reservists 
in  conjunction  with  both  active  duty  and  inactive  duty  training  orders.  As  such,  the 
system  crosses  two  major  Department  of  Defense  (DoD)  functional  areas:  Manpower, 
Personnel  and  Training  (MPT),  and  Financial  Management  (FM). 

B.  HISTORY  OF  RESFMS 

In  the  1970s,  the  Naval  Reserve  operated  an  information  system  for  issuing  active 
duty  orders,  called  Order  Writing,  and  a  system  for  accounting,  called  RPN  Accounting, 
as  two  distinct  systems  operating  on  separate  mainframe  computers.  There  was  no 
interface  between  these  systems.  Information  from  one  system  was  manually  transferred 
to  the  other.  In  1979  the  Naval  Reserve  experienced  an  over-obligation  of  the  RPN 
account  due  to  poor  management  of  active  duty  order  issue  and  travel  expenses.  An 
over-obligation  of  a  Congressional  appropriation  is  prohibited  by  law  under  Title  31 
United  States  Code  1517,  and  may  incur  serious  consequences  for  the  person  responsible 
such  as  suspension  without  pay,  removal  from  office,  fines,  or  imprisonment  (Practical 
Comptrollership,  1992).  As  a  result,  Congress  mandated  in  1980  that  the  Naval  Reserve 
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would  develop  an  information  system  to  correct  the  accounting  and  order  writing 
problems,  that  no  more  over-obligations  would  occur,  and  that  the  new  system  must  be 
operational  by  1984  (Lacy,  1992). 

Staff  members  of  COMNAVRESFOR  assigned  to  Code  10,  the  computer  systems 
management  division,  decided  the  new  system  would  have  three  integrated  subsystems: 
Active  Duty  Order  Writing,  Travel,  and  RPN  Accounting.  They  decided  on  an 
incremental  development  approach.  Navy  Regional  Data  Automation  Center 
(NARDAC),  New  Orleans  was  contracted  to  develop  and  run  the  system  on  their  Sperry 
1100/90  mainframe  computer.  They  wrote  the  programs  in  COBOL  and  used  a 
proprietary  hierarchical  database  provided  by  Sperry  called  DMS1 100.  The  Active  Duty 
Order  Writing  module  was  first  operational  in  February  1983,  and  the  Travel  module  in 
April  1984.  In  1983  NARDAC  informed  the  Naval  Reserve  that  they  would  be  unable 
to  produce  the  entire  system  and  meet  the  1984  deadline.  NARDAC  suggested  that  a 
civilian  contractor  be  hired  to  do  the  RPN  Accounting  module.  Therefore,  the  Naval 
Reserve  contracted  with  CACI,  Inc.  who  subcontracted  with  SYSCON,  Inc.  to  produce 
the  RPN  Accounting  module.  RPN  Accounting  was  first  operational  in  October  1984. 
Upon  system  completion,  SYSCON,  Inc.  was  contracted  to  provide  software  maintenance 
and  NARDAC  provided  hardware  maintenance  arid  support.  Initial  design,  development, 
and  implementation  of  the  system  cost  nine  million  dollars.  Since  1984,  the  company 
providing  contract  software  maintenance  has  changed  twice.  Total  system  costs  through 
1990  were  in  excess  of  $50  million  including  both  investment  and  operating  expense 
(Blaylock,  1990). 
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In  January  1987  RESFMS  became  the  first  Navy  information  system  to  be  certified 
as  compliant  with  requirements  prescribed  for  federal  agencies  by  the  Office  of 
Management  and  Budget  (OMB),  General  Accounting  Office  (GAO),  Department  of  the 
Treasury,  Department  of  Defense  (DoD),  and  Department  of  the  Navy  (DoN).  Very  few 
of  the  121  Navy  accounting  systems  have  achieved  this  certification. 

C.  EVOLUTION  OF  RESFMS 

Since  its  inception,  seven  major  additional  functions,  and  four  major  system 
interfaces  have  been  added  to  RESFMS  (Lacy,  12  August  1991).  Many  small  features 
have  been  added  as  well.  These  additions  will  be  discussed  in  the  following  section  on 
current  configuration.  Considerable  effort  has  also  been  directed  toward  software 
maintenance  which  has  modified  the  structure  and  size  of  many  programs  considerably. 
This  maintenance  effort  has  been  needed  both  because  of  changes  in  the  operating 
environment  and  because  of  the  methods  and  procedures  which  were  used  in  initial 
software  development. 

1.  Changes  in  Operating  Environment 

In  May  1986,  Burroughs  acquired  Sperry  (Barbetta,  1986).  The  merged 
company,  called  Unisys,  offered  a  hardware  upgrade  to  a  new  machine,  the  Unisys 
1 100/92.  This  upgrade  was  installed  in  NARDAC  New  Orleans  in  the  late  1980s.  The 
92  is  similar  to  the  90  but  has  two  CPUs  instead  of  one.  The  two  processors  can  operate 
as  a  tightly  coupled  pair,  essentially  doubling  the  computing  power  of  7.5  MIPS  to  a 
rating  of  15  MIPS.  The  processors  can  also  be  de-coupled  in  software  and  operate  as 
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two  separate  computers  sharing  the  same  peripherals.  Minor  changes  in  code  were 
required  to  run  RESFMS  on  the  upgraded  machine. 

More  important  to  the  maintenance  effort  was  the  fact  that  a  number  of 
additional  functions,  features  and  interfaces  were  added  to  the  system  between  1984  and 
1991.  For  example,  additional  functions  included  order  generation  for  Health  Sciences 
Education  Command  (HSTEC),  issuance  of  travel  claim  vouchers,  and  calculations  using 
the  Uniform  Chart  of  Accounts  (UCA).  Features  added  included  Three  Minute  Orders 
and  batch  printing  of  active  duty  orders  (Lacy,  12  August  1991).  New  interfaces  were 
also  added  with  Micro  Claims  Processing  System  (MCPS),  Integrated  Disbursing  and 
Accounting  Financial  Management  System  (IDAFMS),  Centralized 
Expenditures/Reimbursement  Processing  System  (CERPS),  and  Navy  Standard  Claimancy 
Accounting  Module  (NSCAM)  (CNRF  RESFMS  Briefing  Notes,  June  1991).  These 
additions  required  numerous  software  patches.  Since  they  were  not  part  of  the  original 
design,  integration  and  debugging  were  very  difficult. 

2.  Problems  of  Initial  Software  Development 
a.  Analysis  and  Design 

Inadequate  requirements  analysis  and  product  design  were  performed 
before  coding  initially  began  in  1981.  NARDAC  adopted  a  "code  and  fix"  approach  to 
development,  believing  that  there  was  insufficient  time  to  do  a  thorough  job  of 
requirements  analysis  and  design  prior  to  coding.  One  result  of  the  lack  of  adequate 
design  is  that  more  errors  are  included  in  the  code.  If  coding  begins  without  a  clear  idea 
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of  where  the  project  is  headed  and  exactly  how  to  get  there  (the  result  of  detailed 
design),  many  false  starts  are  made  on  segments  of  program  code,  not  all  of  which  are 
removed  during  debugging.  Even  the  debugging  process  suffers  from  poor  design. 
Without  a  detailed  product  design,  detailed  test  plans  cannot  be  generated  that  fully  test 
all  program  segments  and  functions.  The  result  is  that  the  system  still  includes  many 
undiscovered  errors  even  after  it  is  made  operational  and  those  errors  may  be  difficult 
to  find  because  of  the  poor  structure  and  design.  Such  is  the  case  with  RESFMS  (Lacy, 
30  June  1992).  Therefore,  the  Active  Duty  Order  Writing  and  Travel  modules  have  been 
very  difficult  to  maintain. 

b.  Redundant  Code 

Since  coding  was  begun  without  a  clear  idea  of  overall  product  design, 
procedures  that  should  have  been  identified  as  common  to  many  parts  of  the  application 
were  not.  As  a  result,  when  sections  of  the  program  were  encountered  with  similar 
function  to  those  previously  coded,  large  sections  of  code  were  copied  and  slightly 
modified  to  fit  the  new  situation.  Modem  programming  practice  and  structured  design 
principles  would  have  these  common  procedures  located  in  a  single  common  module 
using  changing  input  parameters  to  produce  the  variations  of  output  (Pressman,  1992). 
Thus,  if  system  maintenance  dictates  that  the  procedure  needs  to  be  modified,  it  can  be 
easily  found  and  changed  in  one  location  which  results  in  the  required  modification  to 
the  entire  system.  In  RESFMS  the  opposite  is  true.  In  order  to  change  one  particular 
function,  all  instances  of  a  segment  throughout  the  application  must  be  found  and 
changed.  This  has  created  numerous  problems  for  maintenance  programmers. 
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c.  Database  Design 

The  database  was  not  well  designed.  It  was  not  adequately  normalized, 
was  created  piecemeal,  contained  redundant  elements  with  different  names,  and  contained 
data  never  used  by  the  application. 

d.  Global  Variables 

Global  variables  were  also  common.  With  many  different  modules 
acting  on  the  same  common  variables,  maintenance  programmers  often  found  that  small 
changes  in  one  module  had  unplanned  and  unwanted  effects  throughout  the  system. 

3.  The  Maintenance  Challenge 

When  Systems  Engineering  and  Management  Associates,  Inc.  (SEMA)  took 
over  as  software  maintenance  contractor  for  RESFMS  in  1989,  they  inherited  a  system 
that  had  been  poorly  designed  and  coded  with  little  structure,  redundant  code,  redundant 
data  elements,  little  structure,  and  prone  to  errors.  Because  of  both  poor  programming 
practices  and  addition  of  functions  and  interfaces,  by  1989  RESFMS  had  grown  in  size 
to  include  (Lacy,  12  August  1991): 

•  Over  two  million  lines  of  COBOL  code 

•  4,500  COBOL  programs 

•  250  record  types  in  database 

•  15.5  million  records  in  database 
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Even  though  the  system  was  fully  functional  and  had  met  stringent  audit  standards,  the 
continued  discovery  of  errors  and  the  addition  of  new  functions  and  interfaces  made  it 
a  maintenance  nightmare. 

4.  The  Maintenance  Solution 

The  COMNAVRESFOR  program  manager,  Ms.  Coreen  Lacy,  mandated  that 
for  the  first  eight  months  of  the  new  contract  no  software  coding  changes  were  to  be 
made.  She  tasked  SEMA  with  a  thorough  analysis  of  the  system,  to  ensure  that,  when 
changes  and  fixes  were  eventually  made,  the  maintenance  programmers  would  fully 
understand  how  the  system  worked  and  what  effect  their  changes  would  make  (Lacy, 
1992).  In  addition,  a  configuration  control  system  was  implemented  to  track  Automated 
Data  Service  Requests  (ADSR)  and  an  error  identification  and  tracking  system  established 
using  Problem  Tracking  System  Reports  (PTSR). 

5.  The  Result 

This  policy  has  paid  great  dividends.  Ms.  Lacy  (30  June  1992)  reports  that 
errors  have  decreased  and  lines  of  code  have  been  reduced  from  a  high  of  over  two 
million  to  about  1.36  million  lines  of  executable  COBOL  code.  More  importantly  for 
this  analysis,  thorough  analysis  of  the  current  system  has  allowed  SEMA  personnel  to 
do  requirements  analysis,  database  redesign,  and  product  design  for  a  proposed  re¬ 
engineered  RESFMS. 
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D.  CONFIGURATION  AND  FUNCTION  OF  RESFMS 

Currently  RESFMS  contains  four  integrated  subsystems: 

•  Active  Duty  Order  Writing  (AT/ADT) 

•  Inactive  Duty  Training  Travel  (IDTT) 

•  Travel 

•  RPN  Accounting 

The  AT/ADT,  Travel,  and  RPN  Accounting  subsystems  are  fully  implemented  on 
a  UNISYS  1100/92  mainframe  computer  operated  by  Navy  Computer  and 
Telecommunications  Station  (NCTS)  New  Orleans.3  Interface  with  the  central 
mainframe  is  via  microcomputers,  emulating  UNISYS  terminals,  located  at  179  Reserve 
Sites  throughout  the  United  States.  Typically,  a  Zenith  248  computer,  equipped  with  an 
emulator  board,  functions  as  a  dumb  terminal  connected  via  modem  to  leased  telephone 
lines,  operating  at  9,600  bits  per  second  (bps).  These  leased  telephone  lines  feed  directly 
to  the  UNISYS  1 100/92  mainframe  at  NCTS,  New  Orleans. 

IDTT  is  processed  by  stand-alone  modules  on  microcomputers  at  all  of  the  353 
Reserve  sites  throughout  the  United  States.  Processed  data  from  the  IDTT  modules  in 
the  field  is  passed  via  modem  and  dial-up  telephone  lines  to  the  mainframe-based 
subsystems  of  RESFMS  for  further  processing  or  storage. 


3Navy  Regional  Data  Automation  Center,  New  Orleans  (NARDAC)  recently  changed 
its  name  to  Navy  Computer  and  Telecommunications  Station,  New  Orleans  (NCTS). 
The  organization  and  equipment  referred  to  as  part  of  the  original  development  of 
RESFMS  are  the  same,  but  the  name  has  changed. 
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1.  AT/ADT  Subsystem 

The  AT/ADT  subsystem  of  RESFMS  is  currently  used  to  issue  approximately 
300,000  sets  of  active  duty  orders  per  year.  Originally,  the  Active  Duty  Order  Writing 
subsystem  issued  Annual  Training  (AT)  orders  for  only  Selected  Reservists  (SELRES), 
that  is,  those  in  an  active  drilling  status  in  a  reserve  unit.  Currently,  RESFMS  allows 
on-line  request  and  subsequent  printing  of  Annual  Training  (AT)  and  Active  Duty 
Training  (ADT)  orders  for  SELRES,  Individual  Ready  Reserve  (IRR),  and  Health 
Science  Educational  Training  Command  (HSETC)  personnel.  Orders  may  be  requested 
either  individually  or  in  batch.  Program  managers  at  Reserve  Headquarters  approve  the 
requests  then  pass  them  electronically  to  Travel  as  appropriate.  After  travel 
arrangements  have  b'vu  made,  orders  may  be  printed  at  the  requesting  site.  Under 
certain  circumstances,  orders  may  be  printed  within  minutes  of  making  the  request.  This 
feature  is  called  3-minute  Orders. 

In  addition,  AT/ADT  performs  many  management  functions  related  to  issuing 
and  accounting  for  active  duty  orders.  The  system  tracks  and  routes  the  request  for 
orders  through  verification,  approval,  travel  arrangements,  accounting,  and  other 
appropriate  stages  of  order  generation.  Modification  and  cancellation  of  order  requests 
are  also  processed.  Program  managers  are  assisted  through  the  tracking  of  days  of  active 
duty  allotted  (budgeted)  versus  those  obligated.  A  history  of  active  duty  performed  is 
maintained  for  individual  reservists  and  Retirement  Points  are  calculated  and  Ur  .ismitted 
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to  Inactive  Manpower  and  Personnel  Management  Information  System  (IMAPMIS)4. 
Finally,  RPN  accounting  transactions  are  generated  for  action  by  the  RPN  Accounting 
subsystem. 

2.  IDTT  Subsystem 

IDTT  processes  and  generates  approximately  200,000  orders  and 
transportation  requests  per  year  for  SELRES  receiving  training  away  from  their  normal 
drill  site  while  in  a  drill  status  (not  on  active  duty).  Accounting  is  also  provided  for 
IDTT  funds  which  are  part  of  the  RPN  appropriation.  A  budget  operating  target 
(OPTAR)  is  issued  to  field  activities  for  IDTT  expenditures.  The  IDTT  module, 
operating  as  a  stand-alone  system  on  a  microcomputer,  allows  the  local  Reserve 
Commanding  Officer  to  keep  track  of  these  funds  while  issuing  IDTT  orders.  Status  of 
funds  is  periodically  passed  to  the  central  RESFMS  mainframe  via  dial-up  modem  and 
batch  reporting.  The  AT/ADT  module  processes  IDTT  order  information  and  passes 
accounting  data  to  the  RPN  Accounting  module  for  financial  update.  If  airline  travel 
arrangements  are  required  in  conjunction  with  IDTT  orders,  the  request  for  travel  is 
transmitted  to  RESFMS  Travel  vial  dial-up  modem  for  processing.  Travel  arrangements 
are  made  and  tickets  disseminated  in  a  manner  similar  to  that  used  for  travel  with  active 
duty  orders. 


4IMAPMIS  runs  on  mainframe  computers  at  Defense  Finance  and  Accounting  Center 
(DFAS),  Cleveland,  Ohio.  It  contains  the  master  records  of  all  Reserve  personnel  and 
financial  information.  Among  other  functions,  IMAPMIS  is  used  to  issue  reserve  drill 
pay  checks  mailed  by  DFAS  Cleveland  to  individual  reservists  or  transmitted  to  their 
bank  accounts.  Retirement  and  promotion  information  is  also  retained  there. 
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3.  Travel 


The  Travel  subsystem  processes  about  167,000  travel  arrangements  and  60 
million  dollars  of  associated  bills  per  year  for  travel  associated  with  both  active  duty  and 
IDTT  orders.  Commercial  airline  reservations  and  ticketing  are  arranged  through 
Scheduled  Airline  Ticket  Office  (SATO).  Tickets  can  be  electronically  transmitted  to 
teleticketing  machines  at  44  major  Reserve  sites,  or  to  airlines  themselves  for  issue.  A 
Government  Transportation  Request  (GTR)  or  a  Request  for  Transportation  Services 
(RTS)  may  be  generated  as  appropriate.  The  system  assists  the  Reserve  Headquarters 
staff  in  selecting  transportation  that  both  meets  operational  needs  and  is  the  lowest  cost 
which  can  be  obtained  at  the  time  of  ticketing.  Thus,  transportation  requests  are 
processed  for  Government  Transportation  System  (GTS),  or  Military  Airlift  Command 
(MAC)  flights,  for  commercial  airline  flights,  charter  bus,  commercial  bus,  and  rail 
transport. 

An  important  benefit  of  the  Travel  subsystem  is  that  it  supports  Ticketing 
Adjustment  and  Unused  Ticket  Recoupment.  Thus,  if  a  ticket  is  issued  for  official  travel 
and  not  used,  the  Travel  subsystem  allows  expeditious  cancellation  of  the  ticket, 
recoupment  of  funds,  and  reallocation  of  resources.  Over  ten  million  dollars  in  travel 
funds  were  recouped  and  reused  in  FY91  alone  (Lacy,  30  June  1992). 

4.  RPN  Accounting 

Full  financial  accounting  and  fund  execution  management  for  the  $700  million 
Reserve  Personnel  Navy  appropriation  are  provided  by  the  RPN  Accounting  module  of 
RESFMS.  In  addition  to  accounting  transaction  entry  and  processing,  it  provides  general 
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ledger  posting  for  Commitments,  Obligations,  and  Accounts  Payable.  RPN  Accounting 
tracks  the  RPN  appropriation  execution  versus  the  budget  plan.  It  provides  numerous 
reports  for  use  within  the  Naval  Reserve  as  well  as  those  required  by  outside  agencies 
such  as  Financial  Information  Processing  Center  (FIPC)  New  Orleans.  An  on-line  ad 
hoc  inquiry  capability  is  available  to  COMNAVRESFOR  Finance  (Code  06),  Manpower 
(Code  02),  and  to  FIPC  New  Orleans. 

5.  External  Interfaces 

RESFMS  also  supports  interfaces  with  six  other  systems: 

•  Inactive  Manpower  and  Personnel  Management  Information  System  (IMAPMIS). 

•  Micro  Claims  Processing  System  (MCPS). 

•  Reserve  Headquarters  System  (RHS). 

•  Integrated  Disbursing  and  Accounting  Financial  Management  System  (IDAFMS). 

•  Centralized  Expenditures/Reimbursement  Processing  System  (CERPS). 

•  Navy  Standard  Claimancy  Accounting  Module  (NSCAM). 

Figure  1  on  the  following  page  shows  the  current  configuration  of  RESFMS. 

E.  REASONS  TO  REDESIGN 

1.  Size  and  Inefficiency  of  Current  Software  Configuration 

Extensive  software  maintenance  efforts  have  reduced  the  size  of  RESFMS  to 
1,360,000  lines  of  code  (LOC).  However,  the  maintenance  patching  process  combined 
with  an  initial  poor  design  of  some  modules  have  caused  the  system  to  still  be  difficult 
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to  maintain  (Furrey,  7  June  1992).  COMNAVRESFOR  spent  $1,777,000  in  FY90  and 
$1,608,000  in  FY91  on  software  maintenance  for  RESFMS.  Systems  maintenance 
programmers  and  analysts  estimate  that  a  well  designed  rewrite  of  RESFMS  should  only 
be  between  450,000  and  720,000  LOC  or  33%  to  52%  of  the  current  size  (Lazar,  16 
July  1992). 5  Program  managers  estimate  that  such  a  system,  programmed  in  an  easy  to 
maintain  language,  such  as  Ada,  using  modem  programming  practice  and  design,  such 
as  structured  programming  with  loosely  coupled,  functionally  independent  modules, 
would  require  annual  maintenance  costing  less  than  half  of  the  current  software 
maintenance  budget  (Furrey,  7  June  1992). 

2.  Operating  Costs  for  Hardware 

RESFMS  is  still  running  on  the  Unisys  1100/92  provided  by  NCTS  New 
Orleans.  Navy  Industrial  Fund  (NIF)  charges  to  COMNAVRESFOR  for  services 
provided  by  NCTS  were  $3,040,000  for  FY90  and  $3,514,000  for  FY91.  In  addition 
to  believing  that  these  charges  are  excessive,  COMNAVRESFOR  managers  feel  that  the 
service  provided  by  NCTS  should  be  rated  as  very  poor  (CNRF  RESFMS  Briefing 
Paper,  August  1991).  Response  by  NCTS  personnel  to  problems  is  often  slow.  The 
charges  for  service  are  based  on  a  fixed  price  contract  (Blaylock,  1990).  Thus,  even  if 
COMNAVRESFOR  reprogrammed  RESFMS  to  consume  fewer  computer  resources, 
charges  for  NCTS  service  would  not  necessarily  decline  proportionally.  This  leaves 


5The  manner  in  which  these  estimates  were  derived  will  be  explained  in  Chapter  VI 
in  the  discussion  of  costs  considered  for  RESFMS. 
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COMNAVRESFOR  few  alternatives  to  reduce  the  operating  costs  of  RESFMS,  which 
they  have  identified  as  excessive,  if  they  continue  to  use  NCTS  operated  hardware 
support. 

3.  Telecommunication  Costs  and  Inadequacies 

Telecommunication  charges  make  up  almost  a  quarter  of  the  RESFMS  total 
operating  expenses  (CNRF  RESFMS  Budget  Expenditure,  14  May  1992). 
COMNAVRESFOR  paid  $2,253,000  in  FY90  and  $1,991,000  in  FY91  to  AT&T  and 
NCTS  for  communication  charges  related  to  RESFMS.  These  charges  result  from  the 
fact  that  COMNAVRESFOR  leases  dedicated  communication  lines  connecting  179  sites 
across  the  continental  United  States  to  the  Unisys  1100/92  at  NCTS  New  Orleans. 
RESFMS  is  currently  programmed  so  that  microcomputers  at  the  remote  sites  function 
only  as  dumb  terminals  and  all  processing  is  done  centrally  by  the  mainframe.  Every 
menu,  prompt,  screen,  and  report  is  generated  by  the  Unisys  1 100/92  in  New  Orleans 
and  transmitted  over  the  leased  telephone  lines  to  the  remote  sites.  Data  transmission, 
at  9,600  bps,  is  very  slow  by  today’s  standards6  and  the  potential  processing  power  of 
the  remote  microcomputers  is  ignored  in  the  present  configuration.  Also,  175  of  the  354 
Naval  Reserve  sites  are  not  connected  to  RESFMS,  because  the  cost  of  connecting  and 
maintaining  dedicated  data  lines  to  these  sites  is  prohibitive. 


6NAVNET  and  DDN  use  56  kilobits  per  second  lines,  while  FTS  2000  offers  T1 
lines  running  at  1.54  megabits  per  second. 
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4.  Integration  With  Other  Naval  Reserve  Systems 

Since  the  inception  of  RESFMS,  the  Naval  Reserve  has  developed  a  number 
of  new  information  systems.  Sensing  both  the  potential  benefits  and  problems  associated 
with  multiple  system  operation,  in  1986  the  COMNAVRESFOR  Director  of  Information 
Systems  (Code  10)  drafted,  and  the  Commander,  then  RADM  Smith,  adopted  the 
Reserve  Command  Management  Information  Strategy  (RESCOMMIS),  a  comprehensive 
plan  for  the  development,  operation,  and  maintenance  of  information  technology  in  the 
Naval  Reserve.  Part  of  this  strategy  involves  integration  of  all  Reserve  systems  as  well 
as  the  pursuit  of  modem  technologies  and  procedures. 

The  first  system  developed  completely  under  RESCOMMIS  was  the  Reserve 
Standard  Training  Administration  and  Readiness  Support  system  (RSTARS). 
Development  and  implementation  of  RSTARS  has  been  successful  (Rautenberg,  15 
September  1991).  RSTARS  is  a  microcomputer  based  distributed  process  connected  via 
dial-up  modem  to  a  centrally  managed  master  database.  The  master  database  is 
maintained  by  the  Reserve  Headquarters  System  (RHS)  which  runs  on  a  cluster  of  DEC 
VAX  minicomputers  and  uses  a  Sharebase  database  machine  for  data  storage.  This 
hardware  is  physically  located  in  COMNAVRESFOR  Information  Systems  (Code  10) 
facilities  in  east  New  Orleans.  At  each  Reserve  site  throughout  the  U.S.,  a  stand-alone 
module  of  RSTARS  manages  the  personnel  files,  unit  assignments,  promotions, 
attendance  records,  pay  records,  training  requirements,  and  other  administrative  data 
required  for  Selected  Reservists.  Modifications  to  this  data  are  sent,  in  the  form  of 
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change  transmittals,  to  RHS  for  validation  and  storage.  RHS  validates  all  changes,  and 
provides  the  interface  to  other  Navy  MPT  systems  requiring  the  data,  such  as  IMAPMIS. 

The  interface  between  RHS  and  RESFMS  is  not  a  continuous  connection. 
In  spite  of  the  fact  that  much  of  the  personnel  data  stored  in  RSTARS  and  RHS  is  exactly 
the  same  as  that  required  by  RESFMS,  there  is  no  sharing  of  this  data.  Changes  to 
personnel  data  which  affect  RESFMS  operation  are  sent  twice  a  month  from  RHS  as  an 
update  to  the  RESFMS  database.  It  is  not  unusual  for  a  Reserve  site  to  have  newly 
assigned  reservists  who  wish  to  perform  their  annual  active  duty  for  training  (AT). 
Frequently,  personnel  data  required  by  RESFMS  for  order  generation  has  not  yet  been 
received  from  RHS  and  a  headquarters  staff  representative  must  manually  enter  the  data 
into  RESFMS  to  allow  the  request  for  orders  to  proceed.  In  my  last  assignment  I  was 
the  Manpower  Department  Head  at  a  major  reserve  site.  This  lack  of  interface  between 
RESFMS  and  RHS  caused  what  I  considered  to  be  a  significant  burden  on  my 
administrative  support  personnel  trying  to  get  active  duty  orders  for  Selected  Reservists. 
It  is  not  just  a  problem  in  the  view  of  users  in  the  field,  however.  COMNAVRESFOR 
information  systems  personnel  believe  that  the  data  redundancy  between  RESFMS  and 
RHS  creates  problems  with  data  integrity  and  consumes  computer  mass  storage  capacity 
that  could  be  better  used  for  other  Naval  Reserve  applications  (Furrey,  14  May  1992). 

F.  DECISION  TO  REDESIGN 

COMNAVRESFOR  managers  decided  in  1991  that  a  redesign  of  RESFMS  should 
be  seriously  examined.  As  a  result  of  their  success  with  RSTARS,  they  believed  a 
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similar  configuration  could  be  employed  with  RESFMS.  The  active  duty  order  request 
process  could  be  reprogrammed  as  a  module  of  RSTARS,  or  at  least  be  compatible  with 
RSTARS.  Using  the  power  of  microcomputers  already  located  at  reserve  sites,  request 
entry,  error  checking,  and  request  formatting  could  all  be  done  off-line.  The  formatted 
request  could  then  be  transmitted  as  a  transaction  via  dial-up  modem,  or  sent  over  a 
packet  switching  network  such  as  the  Defense  Data  Network  (DDN).  As  a  result, 
dedicated  data  lines  would  no  longer  be  needed  and  the  telecommunication  lease  expenses 
could  be  saved. 

Experience  with  RHS  has  also  shown  that  a  large,  data-intensive  process  could  be 
programmed  to  run  on  a  minicomputer  while  maintaining  acceptable  system  performance. 
They  concluded  that  the  power  of  a  mainframe  was  no  longer  required  for  RESFMS  and 
that  a  network  of  minicomputers  might  provide  a  more  cost  effective  replacement.  In 
early  1991  study  of  Local  Area  Network  (LAN)  technology  and  minicomputers  had  been 
initiated  to  seek  a  solution  for  other  Naval  Reserve  requirements  and  to  meet  goals  set 
in  RESCOMMIS.  As  a  result  of  that  study,  a  decision  was  made  to  purchase  six  AT&T 
3B2/600G  minicomputers  off  government  contract  and  establish  an  ethemet  LAN  at 
Naval  Reserve  headquarters  in  New  Orleans.  Experience  with  these  computers  and  this 
LAN  indicated  that  the  3B2/600G  performed  just  as  well  or  better  than  the  VAX  cluster 
running  RHS.  (Albro,  7  April  1992) 

In  April  1991  AT&T  acquired  NCR  (Karpinski,  22  April  1991).  The  newly 
merged  company  began  offering  NCR’s  computing  technology  as  upgrades  to  AT&T 
computers  (Zipper,  10  December  1990)(NCR  letter,  22  June  1992).  AT&T  had  already 
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established  its  3B2/600G  on  government  contract,  and  so  added  the  3B2/GR  as  an 
upgrade  to  that  computer  on  the  contract.  The  AT&T  3B2/GR  runs  a  RISC  based 
processor  and  the  UNIX  operating  system.  It  provides  significant  performance 
enhancements  over  the  3B2/600G.  COMNAVRESFOR  Code  10  made  plans  to  upgrade 
all  COMNAVRESFOR  3B2/600G  computers  to  the  3B2/GR  configuration  as  soon  as 
possible  (Albro,  20  May  1992). 

G.  AUTHOR’S  INVOLVEMENT 

In  September  1991,  I  was  asked  to  assist  the  Naval  Reserve  by  performing  an 
economic  analysis  of  alternatives  for  RESFMS  to  possibly  be  used  in  budget  proposals 
requesting  funds  for  redesign  and  system  acquisition.  The  goal  of  the  redesign  was  to 
reduce  telecommunication  costs,  reduce  system  maintenance  costs,  improve  functionality, 
and  improve  system  integration  with  other  Reserve  information  systems.  Possible 
alternatives  were: 

•  Maintain  the  status  quo 

•  Continue  processing  on  the  Unisys  1100/92  but  change  the  telecommunication 
interface 

«  Redesign  and  recode  the  entire  system  to  run  on  a  network  of  minicomputers  and 
use  microcomputers  as  front  end  processors  in  the  field 

The  target  system  for  complete  redesign  was  to  be  the  AT&T  3B2/GR  minicomputer  in 
order  to  maintain  compatibility  and  consolidated  maintenance  support  with  minicomputers 
already  in  use. 
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H.  TFT  ^COMMUNICATION  COSTS  NO  LONGER  A  FACTOR 

In  April  1992  I  learned  of  a  decision  by  CNRF  management  that  changed  the 
alternatives  to  be  studied  for  economic  analysis.  Approval  had  been  received  for  a 
proposed  Reserve  Data  Communications  Technology  Upgrade.  This  proposal  involves 
the  establishment  of  ethemet  LANs  at  33  major  reserve  sites.  The  33  sites  are 
strategically  located  and  suitably  equipped  so  that  the  other  320  small  reserve  sites  can 
connect  to  the  LANs  by  use  of  a  dial-up  modem  and  regular  telephone  line.  These  LANs 
will  be  connected  to  a  long-haul  communications  carrier  via  an  AT&T  3B2/GR 
minicomputer  operating  as  a  gateway.  The  3B2  runs  UNIX  as  its  operating  system  and 
the  TCP/IP  network  protocols  are  built  into  UNIX,  thus  negating  the  need  to  purchase 
additional  network  management  software.  The  long-haul  communications  are  to  be 
provided  by  NAVNET,  a  packet  switching  network  connecting  U.S.  Navy  commands. 
The  LANs  will  improve  interconnectivity  at  each  site  and,  via  NAVNET,  will  assure 
communication  with  headquarters.  NAVNET  was  chosen  because  it  provides  the 
functionality  which  the  1991  COMNAVRESFOR  LAN  study  determined  was  required 
to  meet  RESCOMMIS  data  needs,  and  it  is  centrally  funded,  that  is,  no  usage  charges 
will  be  made  to  CNRF  for  packets  transmitted  on  NAVNET.  Communications  expenses 
will  be  incurred  only  for  the  links  from  the  Reserve  site  to  the  nearest  NAVNET 
gateway. 

Knowing  of  the  LAN  project  approval  and  NAVNET  communications  capability 
soon  to  be  available,  Mr.  Tom  Albro  of  CNRF  began  investigating  the  potential  for 
connecting  users  to  RESFMS  on  the  Unisys  1100/92  using  NAVNET  and  TCP/IP 
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protocols  (Albro,  7  April  1992).  Experiments  conducted  in  May  1992  confirmed  that 
completely  satisfactory  connectivity  and  functionality  could  be  achieved  using  TCP/IP 
protocols  on  a  packet  switching  network  with  little  or  no  change  to  current  RESFMS 
software  (Albro,  20  May  1992).  Therefore,  a  large  portion  of  the  telecommunications 
costs  for  RESFMS  can  be  eliminated  with  no  software  redesign  and  no  more  hardware 
purchase  than  that  already  approved  for  the  LAN  project.  This  option  will  be  pursued 
regardless  of  any  decision  on  RESFMS  redesign.  Since  reduction  in  communication 
charges  will  be  the  same  for  both  the  status  quo  and  complete  redesign,  any  consideration 
of  costs  or  benefits  associated  with  data  communication  are  now  irrelevant  to  this 
analysis. 

I.  MAINFRAME  TO  MINICOMPUTER  DECISION 

One  significant  hardware  configuration  question  remains.  Given  the  size  and 
complexity  of  RESFMS,  can  it  be  moved  to  minicomputers?  If  yes,  how  many 
minicomputers  will  be  required  to  provide  acceptable  capacity  and  performance? 

In  order  to  answer  these  questions  a  measure  of  performance  and  capacity  had  to 

be  found  that  could  be  meaningfully  applied  to  both  computer  configurations  and  to  the 

process  as  it  currently  runs.  This  kind  of  comparison  is  difficult. 

It  is  almost  impossible  to  make  general  statements  about  different  configurations 
of  hardware  running  under  different  operating  systems.  Only  the  grossest  kinds  of 
comparisons  can  be  made.  Happily,  it  is  really  only  the  grossest  kinds  of 
comparisons  that  are  necessary  to  determine  whether  there  are  significant  hardware 
cost  differences  between  different  machine  system  populations.  (Lorin,  1988) 
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Although  a  thorough  analysis  would  require  benchmark  testing  of  CPU 
performance,  knowledge  of  memory  operations,  page  size,  paging  algorithms,  and  other 
details  of  system  I/O,  an  adequate  comparison  could  be  made  with  simpler  measures 
(Suh,  1991).  It  was  determined  that  a  comparison  of  five  measures  would  be  sufficient: 

•  CPU  performance  measured  in  MIPS 

•  Amount  of  configured  memory  (RAM)  measured  in  megabytes 

•  I/O  throughput  measured  in  bits  per  second 

•  Mass  storage  capacity/requirements  measured  in  megabytes 

•  Number  of  simultaneous  users  supported/required 

Performance  data  for  the  Unisys  1100/92  and  for  RESFMS  was  obtained  from 
NCTS  Pensacola,  which  maintains  all  performance  data  for  NCTS  computers  throughout 
the  southeast  United  States.  Sales  and  technical  representatives  for  AT&T  were 
contacted  to  obtain  performance  and  configuration  data  for  the  3B2. 

The  Unisys  1 100/92  has  two  processors,  each  rated  at  7.5  MIPS.  The  processors 
may  be  coupled  together  to  yield  a  combined  processing  capacity  of  15  MIPS.  The  1 100 
at  NCTS  New  Orleans  has  16  Megabytes  of  RAM,  8  Megabytes  (MB)  of  which  is 
configured  for  RESFMS.  Maximum  I/O  throughput  is  5.0  Megabits  per  second  (Mbps). 
Mass  storage  may  be  attached  in  increments  of  1 .61  Gigabytes  (GB).  Maximum  capacity 
depends  on  number  of  disk  drives  installed.  Operators  claim  the  system  can  support  up 
to  1024  simultaneous  TCP/IP  users.  A  comparison  with  the  AT&T  3B2/GR  is  provided 
in  Table  I  below. 
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The  AT&T  3B2/GR  is  rated  at  25  MIPS.  It  may  be  configured  with  either  32  MB 
or  64  MB  of  RAM.  COMNAVRESFOR  is  purchasing  computers  configured  with  64 
MB  of  RAM.  I/O  bus  speed,  and  therefore  the  maximum  I/O  throughput,  is  5.0  Mbps. 
Each  3B2/GR  may  be  configured  with  up  to  50  GB  of  mass  storage.  However, 
COMNAVRESFOR  is  purchasing  3B2/GRs  with  a  maximum  of  15  GB  per  computer. 
Each  3B2/GR  can  support  up  to  256  simultaneous  TCP/IP  connections  and,  with  UNIX, 
multiple  computers  can  be  connected,  increasing  the  number  of  simultaneous  users 
supported  in  multiples  of  256.  RESFMS  performance  statistics  for  the  period  September 
1991  through  December  1991  show  that  RESFMS  never  used  more  than  12%  of  the 
Unisys  1100/92  CPU  capacity.  Mass  storage  for  RESFMS  required  between  11.0  GB 
and  13.8  GB.  The  number  of  simultaneous  users,  with  the  present  software  architecture 
and  configuration,  was  as  many  as  340. 

Table  I  COMPARISON  OF  UNISYS  1 100/92  AND  AT&T  3B2/GR 


Unisys  11/92 

AT&T  3B2/GR 

Processor  Speed 

15  MIPS 

25  MIPS 

RAM 

16  MB 

64  MB 

Maximum  I/O  Throughput 

5.0  Mbps 

5.0  Mbps 

Hard  Disk  Size 

1.61  GB 

1.2  GB 

Maximum  Mass  Storage 

Unknown 

50  GB 

Mass  Storage  Configured 

15  GB 

15  GB 

Simultaneous  Users 

1024 

256 
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From  the  above  statistics  it  appears  that  the  3B2/GR  is  more  capable  than  the 
Unisys  1 100/92  in  processing  power  and  speed,  and  it  can  be  configured  to  support  I/O 
operations  in  sufficient  quantity  and  speed  to  meet  the  needs  of  RESFMS.  Based  on  the 
analysis  of  current  maintenance  programmers,  it  can  be  assumed  that  a  redesigned 
RESFMS  will  be  smaller  and  not  require  as  much  mass  storage  or  I/O  throughput  as  the 
current  system.  Since  the  3B2/GR  is  adequate  to  the  current  configuration  numbers,  it 
may  be  assumed  it  will  be  more  than  adequate  for  a  redesigned  system.  The  only 
concern  will  be  support  of  simultaneous  users.  Yet,  a  stated  purpose  of  the  redesign  is 
to  change  the  system  architecture  to  take  advantage  of  minicomputer  front-end  processing 
of  data.  This  change  will  greatly  reduce  the  number  of  simultaneous  users  and  negate 
concerns  about  the  3B2/GR’s  ability  to  manage  required  communication  connections. 

Thus,  it  may  be  concluded  that  a  redesigned  RESFMS  could  be  run  on  a  single 
AT&T  3B2/GR  if  necessary  and  that  two  3B2/GRs  running  in  tandem  would  provide 
better  performance  and  reliability  than  the  single  Unisys  1 100/92  does  at  present. 

J.  MOST  SIGNIFICANT  ECONOMIC  FACTOR 

With  the  questions  of  alternative  systems  configurations  and  capabilities  settled,  the 
consideration  of  costs  and  benefits  become  paramount.  The  costs  of  the  current  system 
may  be  readily  obtained  from  COMNAVRESFOR  records.  Most  of  the  costs  of  the 
proposed  redesigned  system  are  also  straightforward.  However,  the  cost  of  redesigning 
and  reprogramming  the  software,  as  we  will  see  in  the  following  chapters,  is  both 
difficult  to  estimate  and  is  crucial  to  this  analysis. 
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IV.  STRUCTURE  OF  ECONOMIC  ANALYSIS 


A.  DEFINING  THE  PROCESS 

The  process  of  economic  analysis  has  been  described  by  Haga  and  Lang  (1991)  as 
"a  systematic,  six  step  procedure  for  comparing  alternative  means  to  meet  an  objective. " 
The  steps  they  define  are  as  follows: 

•  Define  the  Objective 

•  Formulate  Assumptions 

•  Choose  Possible  Alternatives 

•  Compare  Alternatives 

•  Perform  Sensitivity  Analysis 

The  step  of  choosing  possible  alternatives  is  further  divided  into  three  distinct 
activities: 

•  Determine  Costs 

•  Determine  Benefits 

•  Interface  Costs  and  Benefits  for  Each  Alternative 

In  order  to  determine  costs  and  benefits,  we  must  know  how  to  define  them.  In 
order  to  interface,  that  is  compare,  costs  and  benefits  we  must  apply  appropriate 
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economic  analysis  methods.  Although  the  methods  of  comparison  are  similar  for  both 
public  and  private  organizations,  the  analysis  of  public  projects  is  "  con  side,  aoly  more 
sophisticated  than  that  for  private  sector  projects."  (Lang,  1989)  The  reason  public 
project  analysis  is  more  complex  is  because  we  deal  not  only  with  revenues  and  costs, 
as  in  private  projects,  but  also  with  benefits. 

Since  the  economic  analysis  of  RESFMS  concerns  a  U.S.  Government,  and  thus 
public,  project,  it  is  imperative  to  obtain  a  precise  definition  of  costs  and  benefits.  It  is 
also  essential  to  determine  which  economic  analysis  procedures  are  applicable  and  how 
to  apply  them. 

B.  COSTS 

Cost  in  the  public  sector  can  be  defined  as  "a  cash  expenditure  for  operating, 
maintaining,  and  administering  a  public  project."  (Lang,  1989)  Cost  may  also  be  viewed 
as  inputs  or  flows  of  resources  into  the  project,  whereas  benefits  are  outputs  or  results 
of  the  project  (Haga  and  Lang,  1991). 

It  is  important  in  any  economic  evaluation  to  include  all  potential  costs  of  each 
alternative  in  the  analysis.  DoD  Instruction  7041.3  directs  that  in  evaluations  of 
alternatives  for  projects  within  DoD  "costs  of  each  alternative  will  be  exhaustive".  It 
goes  on  to  delineate  three  categories  of  costs  to  be  included: 

•  Research  and  Development  (R&D) 

•  Investment  Costs 

•  Recurring  or  Operations  Costs 
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These  categories  of  costs  apply  to  all  types  of  development  and  acquisition  projects 
within  DoD.  The  project  which  is  the  subject  of  this  analysis,  the  redesign  of  RESFMS, 
involves  the  conversion  of  an  information  system  to  new  equipment  and  software.  In  the 
case  of  projects  involving  replacement,  augmentation  or  conversion  of  existing 
information  processing  assets,  the  FIRMR  directs  that  any  cost  that  can  be  stated  in 
dollars  shall  be  included  with  four  notable  exceptions  that  shall  not  be  included  (FIRMR, 
1990): 7 

•  Conversion  of  existing  software  and  databases  that  would  be  redesigned  regardless 
of  whether  or  not  augmentation  or  replacement  systems  are  acquired 

•  Purging  duplicate  or  obsolete  software,  databases  and  files 

•  Development  of  documentation  for  existing  application  software 

•  Improvements  in  management  and  operating  procedures 

Since  the  proposed  redesign  of  RESFMS  will  be  a  conversion  project,  these  guidelines 
can  be  used  to  determine  costs  applicable  to  this  analysis. 

In  summary,  costs  for  this  analysis  should  be  exhaustive,  should  include  R&D, 
Investment,  and  Operations  costs  and  should  be  expressed  in  terms  of  dollars.  Software 
related  costs  that  will  be  incurred  regardless  of  the  development  decision  are  not  to  be 
included  in  this  analysis,  nor  are  costs  for  improved  management  and  operating 
procedures. 


7Examples  of  costs  that  should  be  included  can  be  found  in  FIRMR  Bulletin  C-14. 
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C.  BENEFITS 


In  private  sector  project  analysis,  the  negative  value  of  costs  can  be  compared  to 
the  positive  value  of  revenues,  or  to  the  market-value  of  potential  system  products  or 
results.  Many  public  sector  projects  have  few  or  no  revenues  to  evaluate  and  the 
products  or  results  of  system  operation  are  not  traded  in  the  marketplace,  making  a 
market-value  approach  useless  (Quirin  and  Wiginton,  1981).  Public  sector  project 
alternatives  can  be  evaluated  on  their  relative  costs,  but  cost  alone  does  not  always 
provide  an  accurate  analysis  of  system  desirability.  Therefore,  a  comparison  of  costs  to 
benefits  is  often  required. 

Lang  (1989)  defines  the  benefit  of  a  public  sector  project  as  "a  cash  advantage  or 
other  favorable  consequence  flowing  to  the  public."  He  goes  on  to  say  that  benefits  "are 
not  difficult  to  identify,  but  are  relatively  difficult  to  quantify  and  to  price."  In  this 
context,  then,  benefits  could  be  described  not  only  as  outputs  but  also  as  "synonymous 
with  results,  effectiveness,  utility,  or  performance."  (Haga  and  Lang,  1991). 

Many  benefits  may  be  assigned  cash  values.  For  instance,  the  construction  of  a 
bridge  may  reduce  the  number  of  miles  driven  by  commuters  each  day  and  thus  save  on 
the  consumption  of  gasoline.  Given  the  number  of  commuters,  average  miles  of  driving 
saved,  average  fuel  consumption,  and  cost  of  gasoline,  these  savings  to  the  public  can 
be  quantified  in  specific  money  terms.  Reducing  the  commuter  miles  driven  daily  may 
also  reduce  the  amount  of  air  pollution  in  the  city,  making  the  air  cleaner  and  the  city 
a  more  pleasant  place  to  live.  How  does  one  assigne  dollar  value  to  a  "more  pleasant 
place  to  live?”  The  answer  may  be  that  a  monetary  value  cannot  be  assigned,  that  this 
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benefit  is  what  Lang  (1989)  calls  an  irreducible  benefit.  If  so,  then  a  means  of 
evaluation  other  than  monetary  valuation  must  be  found. 

A  number  of  techniques  have  been  developed  to  compare  and  evaluate  benefits  in 
public  sector  projects.  One  method  is  to  use  Benefit  Cost  Ratio  (Walker,  1991).  Haga 
and  Lang  (1991)  provide  a  methodology  for  both  quantifying  benefits  and  for  dealing 
with  non-quantifiable  output  measures.  DODI  7041.3  describes  a  method  of  graphical 
comparison  of  benefits  and  costs.  These  methods  are  all  effective  ways  of  dealing  with 
benefits  that  are  irreducible  to  monetary  terms. 

Yet,  while  in  the  past  almost  all  benefits  were  handled  as  irreducibles,  much  more 
effort  is  being  spent  today  on  costing,  or  estimating  the  monetary  value,  of  benefits 
(Lang,  1989).  In  fact,  the  Department  of  Defense  under  CIM  has  recently  mandated 
that  all  benefits  will  be  expressed  "in  cash  terms  so  that  realization  of  benefits  can  be 
monitored  and  audited"  (DDI  Memo,  July  1991).  If  we  express  benefits  in  cash  terms, 
we  do  not  require  any  extraordinary  method  of  economic  analysis  or  comparison.  More 
importantly  from  the  management  standpoint  of  CIM,  if  we  have  devised  means  to 
quantify  benefits  in  dollar  amounts,  we  can  use  those  means  to  verify  that  the  proposed 
benefits  do,  in  fact,  accrue  from  system  implementation. 

Therefore,  in  the  analysis  of  RESFMS,  any  outputs  of  the  system  which  become 
part  of  the  evaluation  should  be  quantified  in  monetary  terms  in  order  to  satisfy  current 
DoD  directives. 
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D.  DISCOUNTING 


As  stated  earlier,  the  concept  of  the  time  value  of  money  is  at  the  heart  of 
economic  analysis.  What  Quirin  and  Wiginton  (1981)  call  the  "bird-in-the-hand" 
principle  states  simply  that  it  is  preferable  to  receive  early  benefits  than  later  benefits. 
The  process  of  discounting  allows  us  to  account  for  this  preference.  Discounting 
calculates  the  present  value  of  a  future  cost  or  benefit.  Present  value  is  obtained  by 
applying  a  discount  factor  to  a  cost  or  benefit. 

In  the  Department  of  Defense,  a  ten  percent  discount  factor  is  to  be  used  when 
evaluating  investment  projects  (DODI  7041.3  and  OMB  Circular  A-94).  Although 
established  in  1972,  this  rate  is  still  considered  to  be  representative.  It  is  an  estimate  of 
the  average,  pre-tax,  rate  of  return  on  private  investment,  after  adjusting  for  inflation. 
Thus,  the  ten  percent  discount  rate  may  be  considered  as  "the  weighted  average 
opportunity  cost  of  taking  money  from  the  private  sector."  (Haga  and  Lang,  1991) 

When  evaluating  investment  decisions  in  the  Department  of  Defense  we  apply 
discounting  by  following  a  two  step  process.  First,  make  all  estimates  of  the  costs, 
savings,  and  benefits  in  terms  of  constant,  base  year  dollars.  Second,  compute  the 
present  value  of  all  cash  flows  by  applying  a  ten  percent  discount  factor.  (Haga  and 
Lang,  1991) 

E.  ECONOMIC  LIFE 

The  length  of  time  over  which  a  project  will  be  evaluated,  the  economic  life,  is  an 
important  factor  in  any  economic  analysis.  A  period  that  is  too  short  may  unfairly 
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penalize  alternatives  that  require  high  initial  investments  and  may  also  hide  the  negative 
effects  of  alternatives  that  have  high  out-year  costs.  Choosing  an  economic  life  that  is 
too  long  may  incorrectly  attribute  value  to  alternatives  that  will  be  obsolete  or  worn  out 
before  the  final  years  of  the  analysis. 

There  are  three  factors  that  determine  economic  life  (Haga  and  Lang,  1991): 

•  Mission  Life  -  The  period  of  anticipated  asset  need. 

•  Physical  Life  -  The  period  the  asset  may  be  used  before  physically  wearing  out. 

•  Technological  Life  -  The  period  the  asset  may  be  used  before  it  becomes 
technologically  obsolete. 

The  economic  life  chosen  for  analysis  of  alternatives  is  usually  the  shortest  of  the 
mission,  physical  and  technological  lives.  (Haga  and  Lang,  1991) 

F.  ANALYSIS  TOOLS 

Of  the  economic  analysis  tools  available  for  public  and  private  sector  investment 
decision  analysis,  there  are  six  techniques  which  are  appropriate  for  evaluation  of 
information  systems.  (Walker,  1991) 

1.  Present  Value  Analysis  (PV) 

This  method  determines  each  alternative’s  costs  as  stated  in  terms  of  their 
present  value.  It  requires  all  alternatives  to  be  of  equal  economic  lives.  This  procedure 
is  to  be  used  when  the  economic  life  of  a  project  is  more  than  three  years  (Haga  and 
Lang,  1991  and  DODI  7041.3). 
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Present  value  analysis  is  the  primary  economic  analysis  tool  and  its 
calculations  become  the  basis  for  calculations  by  other  analysis  tools.  When  costs  and 
benefits  considered  can  be  quantified  in  terms  of  dollars,  present  value  analysis  is  the 
preferred  technique.  Other  economic  analysis  tools  complement  present  value  analysis 
by  providing  means  of  comparing  alternatives  when  they  are  of  unequal  lives,  or  when 
costs  and  benefits  cannot  be  expressed  in  dollars.  Additional  information  is  also  provided 
by  some  techniques  amplifying  the  present  value  analysis  results.  However,  unless  a 
mistake  is  made,  other  methods  will  never  contradict  the  results  obtained  by  present 
value  analysis. 

2.  Uniform  Annual  Cost  (UAC) 

When  evaluating  alternatives  with  unequal  economic  lives,  the  Uniform 
Annual  Cost  method  may  be  used  to  rank  the  alternatives.  Because  it  is  based  on  present 
value  analysis,  if  the  alternatives  evaluated  have  equal  economic  lives,  UAC  is  redundant 
to  present  value  analysis.  (Walker,  1991) 

3.  Savings/Investment  Ratio  (SIR) 

This  ratio  computes  the  relationship  between  future  cost  savings  and  the 
investment  required  to  obtain  those  savings.  "Because  saving  is  a  necessary  ingredient, 
you  use  this  if,  and  only  if,  you  have  a  status  quo  alternative."  (Haga  and  Lang,  1991) 
DODI  7041.3  states  that  the  Savings/Investment  Ratio  should  be  shown  when  evaluating 
cost- reduction  investment  proposals  involving  incremental  costs. 
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4.  Discounted  Payback  (PB) 

Discounted  Payback  simply  measures  the  amount  of  time  it  takes  for  an 
alternative  to  pay  for  itself.  It  determines  what  period  of  i  me  is  required  for  the 
accumulated  present  value  of  cost  savings  to  offset  the  total  present  value  cost  of  an 
alternative.  (Haga  and  Lang,  1991)  Again,  since  savings  are  involved,  this  method  may 
be  used  if,  and  only  if,  there  is  a  status  quo  alternative. 

5.  Break  Even  Analysis  (BE) 

When  an  information  system  will  have  variable  costs  as  well  as  fixed 
investment  requirements,  Break  Even  Analysis  may  be  useful.  Garrison  (1988)  describes 
Break  Even  Analysis  as  finding  the  point  where  a  project’s  total  expenses  equal  its  total 
revenue.  This  point  will  also  be  where  the  decision  maker  will  be  indifferent  to  whether 
the  project  should  be  undertaken  or  not. 

6.  Benefit/Cost  Ratio  (BCR) 

BCR  computes  the  ratio  between  outputs  (benefits)  of  a  project  and  inputs 
(costs).  BCR  can  be  used  to  compare  both  quantitative  and  non-quantitative  benefits. 
Of  the  six  techniques  judged  applicable  to  information  systems,  this  is  the  only  technique 
that  can  be  used  to  evaluate  non-monetary  benefits. 

G.  SELECTION  OF  TOOLS  FOR  RESFMS 

One  of  the  alternatives  for  RESFMS  will  be  a  status  quo  alternative  and  a  chief  aim 
of  the  redesign  is  cost  savings.  Therefore,  both  Saving/Investment  Ratio  and  Discounted 
Payback  would  be  appropriate  analysis  methods.  Since  costs  and  benefits  will  be 


expressed  in  dollars,  Present  Value  Analysis  is  appropriate  and  preferred  as  a  primary 
evaluation  tool.  The  chosen  economic  life  will  be  equal  for  all  alternatives,  which  would 
make  Uniform  Annual  Cost  redundant  to  Present  Value  Analysis.  There  are  no  costs  that 
vary  with  work  load.  Thus  Break  Even  Analysis  is  probably  not  appropriate.  Recently 
published  CIM  policy  (DDI  Memo,  23  July  1991)  directs  that  any  benefits  used  in  an 
economic  analysis  be  valued  in  monetary  terms.  The  primary  purpose  of  the  redesign 
is  cost  savings,  not  added  benefit.  The  benefits  that  will  accrue  are  difficult  to  quantify 
in  monetary  terms.  A  Benefit/Cost  Ratio  might  be  useful,  but  would  be  unacceptable  to 
CIM  reviewers  unless  the  benefits  were  in  monetary  terms.  Techniques  for  comparing 
costs  and  cost  savings  will  probably  be  more  than  adequate  for  the  analysis.  Therefore, 
Benefit/Cost  Ratio  will  not  be  used. 

The  economic  analysis  of  alternatives  for  RESFMS,  then,  will  involve  Present 
Value  Analysis,  Saving/Investment  Ratio,  and  Discounted  Payback  analysis  of  each 
alternative  using  the  standard  DoD  discount  rate  of  ten  percent. 

H.  RISK  ANALYSIS 

The  economic  analysis  process  is,  by  its  nature,  uncertain.  No  matter  how 

conscientious  one  is  in  identifying  and  evaluating  costs  and  benefits,  the  process  must  use 

estimates  and  estimates  involve  uncertainty. 

If  the  economic  evaluation  method  used  does  not  reflect  this  uncertainty,  then  every 
assumption  built  into  an  economic  analysis  is  a  "best  guess"  and  the  final  economic 
result  is  a  consolidation  of  these  "best  guesses."  Making  decisions  on  the  basis  of 
such  "best  guess"  calculations  alone  can  be  hazardous.  (Stermole,  1984) 
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Therefore,  the  last  step  of  the  economic  analysis  procedure  is  an  evaluation  of 
uncertainty,  or  risk  analysis. 

A  number  of  methods  exist  for  risk  analysis.  They  range  from  simple  methods  to 
highly  complex  simulations.  The  method  generally  preferred  in  private  corporations  in 
the  1980s  was  categorical  ranking  of  risk  (Strassmann,  1990).  The  types  of  risk  are 
described  by  adjectives  (such  as  high,  low,  moderate,  disaster).  Then  the  risk  types  are 
converted  to  numeric  scales  by  assigning  weights  to  each  category.  More  elaborate 
evaluations  can  be  made  by  increasing  the  number  of  risk  indicators  to  be  ranked. 

Sensitivity  analysis  is  a  means  of  evaluating  the  effects  of  uncertainty  by  varying 
various  parameters  and  thus  determining  their  effect  on  the  economic  evaluation  results. 
(Stermole,  1984)  First,  computations  are  made  with  the  best  estimate  of  values  for  each 
variable.  Then,  by  changing  the  values  of  variables  within  reasonable  limits  and 
recomputing  the  results,  the  effects  of  each  variable  on  the  final  outcome  can  be  readily 
seen.  (Haga  and  Lang,  1991)  Through  sensitivity  analysis,  critical  strategic  variables  can 
be  identified  for  careful  attention  by  the  decision  maker  and,  if  the  project  is  approved, 
for  close  observation  by  the  program  manager. 

Probably  the  most  sophisticated  method  of  analyzing  risk,  and  one  gaining  in 
popularity,  is  financial  simulation  using  computer  models  (Strassmann,  1990). 
Simulation  requires  that  you  be  able  to  assign  probability  distributions  to  each  major  cost 
determinant  (Haga  and  Lang,  1991).  Several  commercially  available  software  packages 
exist  that  work  on  microcomputers  as  either  stand-alone  programs  or  as  add-in  features 
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to  spreadsheet  programs.8  These  software  packages  generally  use  randomly  generated 
numbers  and  a  Monte  Carlo  method  (such  as  spinning  an  imaginary  roulette  wheel)  to 
simulate  the  probability  of  occurrence  of  costs  and  benefits.  When  the  simulation  is 
finished,  the  relative  frequency  of  the  various  values  can  be  plotted  on  a  chart  or  graph. 
This  form  of  simulation  and  display  of  results  can  be  useful  in  the  analysis  of  project 
risk. 

Sensitivity  analysis  and  risk  analysis  of  economic  evaluations  in  the  Department  of 
Defense  has  been  encouraged  for  more  than  twenty  years  (DODI  7041.3).  However, 
recently  it  has  taken  on  new  import.  The  Director  of  Defense  Information  has  directed, 
as  a  part  of  CIM  policy,  that  calculations  of  cost  and  benefit  for  information  systems 
projects  in  DoD  will  be  adjusted  to  reflect  the  financial  impacts  of  risk  (DDI  Memo,  23 
July  1991).  Therefore,  in  the  analysis  of  alternatives  for  RESFMS,  an  evaluation  of 
financial  risk  of  the  estimates  will  also  be  made. 


8Some  examples  of  simulation  software  packages  for  microcomputers  are:  Risk 
Analysis  and  Simulation  (for  DOS),  and  ©Risk  (add-in  program  for  Lotus  1-2-3),  both 
from  Palisade  Corporation,  Newfield,  N.Y.,  and  Crystal  Ball-Forecasting  and  Risk 
Management,  Market  Engineering  Corporation,  Denver,  Colorado  (for  Macintosh). 
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V.  PROBLEMS  WITH  SOFTWARE  DEVELOPMENT  COSTING 


A.  COST  ESTIMATION  METHODS 

The  cost  of  redesigning  and  reprogramming  the  software  for  RESFMS  is  the  most 
difficult  to  estimate  and  the  most  critical  to  this  analysis  of  all  of  the  costs  associated 
with  the  redesign  alternative.  The  problem  of  devising  accurate  and  reliable  cost 
estimates  for  the  development  of  software  systems  is  not  new,  nor  is  it  unique  to  this 
analysis.  Four  methods  to  estimate  software  development  cost  appear  in  the  literature 
(Hihn  and  Habib-agahi,  1991): 

1.  Price  to  Win 

The  method  dubbed  by  Boehm  (1981)  as  "Price  to  Win"  in  the  private  sector 
involves  making  the  cost  estimate  equal  to  that  believed  necessary  to  win  the  job.  The 
same  reasoning  may  be  applied  in  the  public  sector  to  those  estimates  made  to  equal  the 
amount  (or  schedule)  believed  to  be  desired  (or  politically  acceptable)  by  those  who  will 
approve  the  project. 

2,  Analogy 

This  method  involves  reasoning  by  analogy  with  known  previous  development 
efforts.  The  actual  costs  of  completed  projects  are  related  to  an  estimate  of  the  cost  of 
a  similar  new  project.  Boehm  (1981)  points  out  that  the  major  advantage  of  this  method 
is  that  it  is  based  on  actual  project  experience.  The  disadvantage  is  that  it  is  unclear  to 
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what  degree  the  previous  project  effort  is  actually  representative  of  the  effort  required 
on  the  new  project. 

3.  Expert  Judgement 

When  expert  judgement  is  used,  one  or  more  experts  are  consulted  who  use 
their  experience  and  knowledge  of  the  proposed  project  to  estimate  the  effort,  and  thus 
cost  required.  The  disadvantage  of  expert  judgement  is  that  it  "is  no  better  than  the 
expertise  and  objectivity  of  the  estimator,  who  may  be  biased,  optimistic,  pessimistic, 
or  unfamiliar  with  key  aspects  of  the  project."  (Boehm,  1981)  The  estimate  obtained  by 
expert  judgement  may  also  not  be  repeatable  by  any  other  estimator. 

4.  Algorithmic  Models 

These  methods  use  one  or  more  algorithms  which  produce  an  estimate  of  the 
costs  as  a  function  of  some  number  of  variables  considered  to  be  cost  drivers.  The 
algorithms  may  be  manually  applied  or  incorporated  into  an  automated  costing  tool. 
Boehm  (1981)  describes  the  five  most  common  forms  of  estimation  algorithms: 

•  Linear  models 

•  Multiplicative  models 

•  Analytic  models 

•  Tabular  models 

•  Composite  models 
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All  algorithmic  models  require  some  measure  of  project  size  as  an  input.  The 
advantage  of  algorithmic  models  is  that  they  are  objective,  repeatable,  and  generally 
efficient.  However,  the  estimate  produced  is  no  more  accurate  than  the  sizing  inputs  and 
cost  driver  ratings  used,  and  this  constitutes  the  main  disadvantage  of  algorithmic  models. 
First,  it  may  be  difficult  to  obtain  an  accurate  size  estimate  at  the  time  the  cost  estimate 
is  most  needed.  Second,  the  cost  driver  ratings  used  must  be  validated  and  calibrated  for 
the  particular  organization  doing  the  development.  None  of  these  models  is  generic  in 
the  sense  of  being  able  to  apply  the  model  to  all  development  sites  universally  without 
modification  or  calibration  of  some  sort. 

B.  COCOMO 

Probably  the  most  well  known  and  widely  studied  software  cost  model  is  the 
hierarchy  of  models  called  COCOMO,  for  Constructive  COst  MOdel,  described  by 
Boehm  (1981).  Boehm’s  hierarchy  involves  three  models:  Basic  COCOMO,  Intermediate 
COCOMO,  and  Advanced  COCOMO.  COCOMO  is  an  example  of  an  algorithmic 
model.  Basic  COCOMO  is  a  static,  single-value  model.  It  computes  development  effort 
(in  man-months)  and  costs  (in  dollars)  as  a  function  of  program  size  expressed  in  number 
of  lines  of  code.  Intermediate  COCOMO  expands  on  the  basic  model  by  using  a  set  of 
cost  drivers  to  modify  the  estimate.  These  cost  drivers  are  numeric  values  derived  from 
a  subjective  assessment  of  system,  hardware,  personnel,  environmental,  and  project 
attributes.  Advanced,  or  Detailed  COCOMO  incorporates  all  the  attributes  of 
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Intermediate  COCOMO  and  adds  an  assessment  of  the  cost  driver’s  impact  on  each  phase 
of  the  software  development  process. 

The  phases  of  software  life-cycle  used  in  COCOMO  are  those  of  the  waterfall 
model.  Originally  presented  by  Royce  (1970),  and  widely  used  in  the  1970s  and  1980s, 
the  waterfall  model  has  been  codified  in  U.S.  Government  and  Department  of  Defense 
directives  and  instructions  as  the  primary  means  of  development  and  documentation  of 
information  systems.  The  seven  project  activity  phases  for  which  COCOMO  computes 
effort  are: 

•  Plans  and  Requirements 

•  Product  Design 

•  Programming 

•  Integration  and  Test 

•  Development 

•  Maintenance 

COCOMO  was  developed  as  a  result  of  an  analysis  of  63  software  projects 
completed  by  TRW,  Inc.  (Boehm,  1981).  Many  algorithmic  models  are  described  only 
in  general  terms  because  all  or  part  of  the  model  includes  proprietary  information. 
COCOMO  is  explained  in  detail,  with  reference  to  both  rationale  and  actual  computation 
of  each  aspect  of  the  model.  COCOMO  also  provides  effective  tools  for  software  project 
management  throughout  each  phase  of  development.  Thus,  COCOMO  has  become  the 
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centerpiece  of  software  project  estimation  and  management  instruction,  as  well  as  the 
basis  for  other  automated  estimating  tools. 

C.  SOFTWARE  SIZE  METRICS 

Mcjt  of  the  algorithmic  models  for  cost  estimation  use,  as  their  basic  input,  a 
measure  of  project  size  in  terms  of  lines  of  code  (LOC)  or  thousands  of  lines  of  code 
(KLOC).  LOC  has  also  been  used  extensively  as  a  productivity  measure  to  determine 
project  progress  and  programmer  efficiency.  Those  who  favor  using  LOC  as  a  measure 
claim  that  there  is  high  correlation  between  LOC  and  software  development  costs,  that 
LOC  can  be  easily  counted,  and  that  "a  large  body  of  literature  and  data  predicated  on 
LOC  already  exists."  (Pressman,  1992) 

Another  term  for  LOC,  used  by  Boehm  (1981),  is  delivered  source  instructions 
(DSI),  or  thousands  of  DSI  (KDSI).  The  slightly  different  terminology  suggests  one  of 
the  problems  with  LOC  measures:  although  the  measure  appears  to  be  objective,  the 
definition  of  what  constitutes  one  line  of  code,  or  one  source  instruction,  is  not  always 
clear.  This  and  other  problems  with  the  metric  are  discussed  in  detail  by  Jones  (1986). 
The  use  of  LOC  as  a  metric  also  penalizes  well-designed  but  shorter  programs,  and  does 
not  easily  adapt  to  measuring  nonprocedural  languages  such  as  Fourth  Generation 
Languages  (4GL).  The  most  important  problem  for  cost  estimation,  however,  is  that 
using  LOC  requires  a  level  of  detail  which  may  be  difficult  to  achieve  at  the  time  the 
estimate  is  required  (Pressman,  1992). 


54 


D.  FUNCTION  POINTS 


An  alternative  to  size  metrics  such  as  LOC  is  the  measurement  of  software 
"functionality"  or  "utility."  Function  oriented  metrics  were  first  proposed  by  Albrecht 
(1979)  while  serving  as  Program  Manager  of  the  Application  and  Maintenance 
Measurement  Program  for  IBM  (Behrens,  1983). 

Albrecht’s  basic  premise  was  that  all  the  functions  present  in  an  application  can  be 
measured  by  examining  the  factors  which  are  the  "outward  manifestations  of  any 
application"  (Albrecht,  1979).  The  procedure  is  to,  first,  list  and  count  the  number  of 
external  user  inputs,  outputs,  queries,  and  the  number  of  external  and  internal  master 
files. 

Each  of  these  categories  of  input  and  output  are  counted  individually  and  then 
weighted  by  numbers  reflecting  the  relative  value  of  the  function  to  the 
user/customer.  The  weighted  sum  of  the  inputs  and  outputs  is  called  "function 
points."  (Albrecht  and  Gaffney,  1983) 

The  weighted  sum  is  then  adjusted  for  other  development  environment  factors  such  as 
communication  complexity  and  time  criticality.  A  subjective  judgement  of  these  general 
environmental  factors  is  converted  into  a  numeric  value  called  Degree  of  Influence  (DI) 
which  serves  as  the  basis  for  a  General  Characteristics  Adjustment  (GCA)  to  the  function 
point  count. 

The  final  product  of  this  process  is  a  number  that  can  be  used  as  a  relative  measure 
of  program  functionality  and  complexity.  By  computing  function  points  for  various 
systems  and  comparing  the  amount  of  programmer  and  analyst  effort  required  to  produce 
the  system,  a  metric  can  be  found  to  estimate  project  cost.  For  instance,  a  given 
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organization  may  find  that  average  productivity  is  five  function  points  per  man-month 
effort.  If  a  proposed  project  has  200  function  points,  this  organization  can  expect  to 
expend  40  man-months  producing  the  application.  By  applying  the  locally  determined 
cost  per  man-month,  a  dollar  cost  estimate  can  also  be  made. 

The  advantage  of  function  point  analysis  is  that  a  far  better  picture  of  project 
function  and  complexity  can  be  obtained  than  that  achieved  with  LOC  estimates. 
Function  points  also  give  a  measure  of  programming  effort  independent  of  program 
language  used.  The  following  table  (Pressman,  1992  and  Jones,  1986),  illustrates  this 
fact  by  showing  a  rough  estimate  of  the  average  lines  of  code  required  to  build  one 
function  point  in  various  programming  languages: 

Table  II  AVERAGE  LINES  OF  CODE  PER  FUNCTION  POINT 


Programming  Language 

LOC/FP  (Average) 

Assembly  language 

300 

COBOL 

100 

FORTRAN 

100 

Pascal 

90 

Ada 

70 

OBJECTIVE-C 

25 

Fourth-generation  languages  (4GL) 

20 

Code  generators 

15 
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Examination  of  the  above  data  shows  that  one  LOC  of  Ada  provides  approximately  1.4 
times  the  "functionality"  as  one  LOC  of  COBOL  (Pressman,  1992).  In  other  words,  a 
program  written  in  COBOL  taking  (on  average)  1 ,000  LOC  should  only  take  about  700 
LOC,  or  be  only  70  percent  as  large,  if  rewritten  in  Ada. 

One  disadvantage  of  using  function  points  is  that,  although  it  may  appear  very 
objective  and  straightforward,  counting  of  function  points  involves  subjective  judgements. 
Still,  the  advantages  of  function  points  as  a  software  productivity,  project  estimation,  and 
project  management  tool  are  considered  by  some  to  be  significant  (Behrens,  1983). 

E.  RELIABILITY  OF  MODELS 

Boehm  (1981)  asserted  that  models  such  as  COCOMO  could  be  expected  to  provide 
an  estimate  of  cost  within  20%  of  the  actual  cost  70%  of  the  time.  In  spite  of  great 
effort  to  improve  cost  estimation  and  develop  new  models,  the  statistics  have  not  changed 
much  today.  Stephen  Gross  (1992),  AIS  Division  Head  of  the  Naval  Center  for  Cost 
Analysis,  related  his  work  in  validating  a  new  automated  cost  model  for  DoD  use  called 
SASET  for  Software  Architecture,  Sizing,  and  Estimating  Tool.  He  stated  that  the 
criteria  for  acceptance  of  this  model  was  that  its  predicted  cost  should  be  within  20%  of 
the  actual  system  cost  80%  of  the  time.  To  validate  the  model,  researchers  used 
historical  data  from  25  systems  developed  for  the  Department  of  Defense.  Using  perfect 
information  of  project  size  and  complexity,  as  determined  by  project  completion,  the 
model  could  predict  within  20%  of  the  correct  cost  80%  of  the  time  (Gross,  1992). 
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However,  at  the  time  a  cost  estimate  is  required,  before  the  project  begins,  perfect 
information  of  project  size  and  complexity  is  not  available.  One  can  easily  see  that  if  the 
size  estimate  used  is  70%  accurate  and  the  model  80%  accurate,  we  only  have  a  56% 
chance,  or  slightly  better  than  even  odds,  of  being  within  20%  of  actual  project  cost. 

Therefore,  the  better  our  understanding  of  the  design  of  the  system,  the  more 
accurate  will  be  our  estimate  of  effort.  Actually,  this  statement  is  true  regardless  of  the 
method  used  for  cost  estimation.  If  we  use  an  algorithmic  model,  a  fairly  detailed 
description  of  system  design  will  be  required  to  make  a  reliable  estimate  of  lines  of  code 
required  to  implement  the  system.  If  function  points  are  used,  a  detailed  description  of 
system  requirements  and  an  accurate  estimation  of  all  input  and  output  is  needed  to  make 
the  calculation.  Even  expert  opinion  and  analogy  require  some  knowledge  of  system 
design  for  a  adequate  estimate  to  be  made.  The  dilemma  then  becomes:  how  much 
project  development  does  one  complete  before  making  an  estimate  of  cost  and  effort 
required?  Pressman  (1992)  states  that  no  matter  attractive  it  may  be  to  delay  estimation 
until  very  late  in  the  project,  this  is  not  a  practical  option.  He  suggests  instead  that 
several  techniques  be  used  "in  tandem,  each  used  as  a  cross-check  for  the  others." 
(Pressman,  1992)  This  analysis  of  RESFMS  will  use  just  such  an  approach. 

F.  THE  SOFTWARE  COST  ESTIMATION  PROBLEM  IN  DOD 

Current  DoD  policy  requires  that  a  business  case  or  functional  economic  analysis 
be  done  prior  to  project  approval.  Part  of  the  business  case  is  a  cost/benefit  analysis  for 
the  proposed  information  system.  Project  approval  must  be  received  before  funds  are 
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budgeted  for  the  project.  In  order  to  determine  an  estimate  of  project  costs  and  benefits, 
system  requirements  and  some  level  of  product  design  must  be  performed.  Yet, 
requirements  analysis  and  product  design  are  the  first  two  phases  of  the  project  for  which 
we  are  seeking  approval.  What  then  can  be  done?  Or  better,  what  is  in  fact  done? 

The  FIRMR,  OMB  circulars,  GSA  guides,  and  DoD  directives  all  state  that 

requirements  analysis  as  a  part  of  project  planning  should  begin  as  early  as  possible. 

Agencies  are  encouraged  to  develop  strategic  plans  for  business  activities  and  information 

resource  (IR)  needs.  From  strategic  plans  come  tactical  plans  indicating  specific  IR 

systems  required  for  mission  fulfillment.  When  IR  system  development  needs  are 

identified,  a  program  manager  (PM)  should  be  appointed  and  project  planning  should 

begin  (GSA  Overview  Guide,  1990).  DoD  instructions  reinforce  this  concept.  DoD 

Instruction  7920.2  describes  the  AIS  life-cycle  management  process,  the  milestones 

required  and  activities  in  each  phase.  Request  for  Milestone  0  ends  the  Need 

Justification  Phase  and  involves  submission  of  a  Mission  Needs  Statement  (MNS). 

The  MNS  is  approved  at  Milestone  0,  and  the  DoD  Component  is  authorized  to 
initiate  the  Concepts  Development  Phase  and  to  expend  resources  for  the  activities 
of  that  Phase  as  planned.  (DODI  7920.2,  7  March  1990) 

The  recognition  of  a  need  to  expend  resources  in  planning  for  future  systems 
development  is  not  confined  to  government  publications  and  directives.  Strassmann 
(1990)  states  that  five  percent  of  an  organization’s  information  budget  should  be  directed 
to  development  plans.  Since  Strassmann  is  now  Director  of  Defense  Information  (DDI), 
one  would  think  this  opinion  would  be  strongly  reinforced  in  practice.  Yet,  in  spite  of 
the  encouragement  in  GSA  publications,  the  opinion  of  Strassmann,  and  wording  of  DoD 
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instructions,  DoD  budget  management  personnel  will  not  allow  any  funds  to  be  identified 
explicitly  for  IR  projects  until  that  project  has  been  fully  approved  for  development  or 
redesign. 

This  fact  is  evident  in  examination  of  the  RESFMS  budget.  A  strong  argument  can 
be  made  for  the  need  to  redesign  RESFMS.  Yet,  because  RESFMS  is  considered  a  fully 
deployed  system,  and  no  redesign  has  been  officially  approved,  no  monies  are  designated 
for  any  redesign  activities,  nor  have  they  been  for  the  last  two  years.  If  a  budget  line 
item  for  development  had  any  value  other  than  zero,  Navy  budget  personnel  would  cut 
the  organization’s  total  budget  by  that  amount  and  zero  out  the  line  item.  The  reasoning 
would  be  that  there  was  no  authorization  for  expending  such  funds  on  a  fully  deployed 
system. 

This  mentality  is  not  unique  to  the  Naval  Reserve.  Interviews  with  CAPT  Faubel 
(1992),  .he  comptroller  for  Commander  in  Chief  Atlantic  Fleet,  and  with  David  Spivey 
(1992)  of  the  U.S.  Army  Corps  of  Engineers  confirm  that  the  reasoning  is  the  same  in 
their  organizations  as  well.  Gross,  of  the  Navy  Center  for  Cost  Analysis,  does  not 
believe  that  this  presents  a  problem.  He  believes  that  if  a  commander  feels  a  project  is 
needed  the  commander  will  find  the  money  somewhere  in  his  budget  to  perform  the 
needed  cost  analysis.  He  also  stated  that,  in  his  experience,  major  commands  desiring 
to  develop  systems  had  "plenty  r>f  money"  available  for  this  purpose  (Gross,  1992). 
Others  do  not  agree. 

Both  CAPT  Faubel  (1992)  and  Mr.  Spivey  (1992)  acknowledged  that  it  is  a  serious 
problem  that  money  is  not  officially  available  until  after  a  project  is  approved,  yet  a 
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portion  of  the  first  phases  of  project  development  must  be  done  in  order  to  obtain  good 
cost  estimates  required  for  project  approval.  Neither,  however,  wished  to  see  a  DoD 
requirement  to  explicitly  identify  funds  for  this  purpose  in  a  budget,  and  especially  not 
if  a  specific  figure  or  percentage  were  tied  to  the  requirement.  Both  preferred  to  spend 
funds  out  of  operating  expenses,  under  the  heading  of  general  project  management,  thus 
allowing  more  organizational  discretion  in  the  execution  of  the  budget.  William  Curtis 
(1992)  of  the  Software  Engineering  Institute  (SEI)  stated  that  this  is  indeed  a  problem. 
He  felt  that  it  is  not  just  a  problem  in  the  Department  of  Defense,  but  also  in  industry. 
However,  it  is  not  recognized  as  a  problem  in  industry.  He  felt  that  insufficient 
resources  were  being  devoted  in  general  to  the  activities  required  to  perform  good  cost 
estimates  for  software  development  projects  before  decisions  are  made  to  proceed  with 
those  projects. 

Therefore,  even  though  it  is  widely  recognized  that  resources  should  be  expended 
to  provide  the  information  required  for  a  reasonable  cost  estimate  before  system 
development  approval,  these  resources  cannot  now  be  explicitly  identified  in  DoD 
budgets.  The  assumption  of  this  discussion  is  that  funds  are  available  in  the  program 
management  budget  of  every  agency  to  support  these  activities.  This  assumption  may 
not  be  true  for  all  agencies  needing  to  develop  IR  systems,  especially  not  in  times  of  a 
declining  budget.  In  the  case  of  the  Naval  Reserve,  such  excess  funds  do  not  exist.  The 
Naval  Reserve  may  not  be  unique  in  this  regard. 

Unlike  most  DoD  agencies,  the  U.S.  Army  Corps  of  Engineers  receives  both  an 
appropriated  budget  and  significant  nonappropriated  income  from  services  provided  to 


61 


other  governments  and  outside  agencies.  The  Corps  of  Engineers  has  been  successful 
in  information  system  development  (Spivey,  1992).  It  has  had  a  series  of  excellent 
commanders  who  understood  the  problems  and  advantages  of  information  system 
development  and  who  have  been  supportive  of  expenditures  in  support  of  re-engineering 
efforts  in  the  Corps.  In  spite  of  this  environment,  and  a  long  period  of  increasing 
defense  budgets  in  the  1980s,  Spivey  (1992)  stated  that  getting  money  to  do  proper 
requirements  analysis  and  system  cost  estimates  was  still  a  problem— one  that  they  have 
so  far  overcome,  but  still  a  problem.  If  an  organization  such  as  the  Corps  of  Engineers 
experiences  a  problem  with  this,  what  then  do  agencies  without  the  same  resources  and 
expertise  do? 

G.  RESPONSE  TO  THE  PROBLEM  IN  DOD 

As  a  result  of  my  experience  and  interviews  with  several  information  system 
managers  who  did  not  wish  to  be  quoted,  I  believe  agencies  do  one  of  several  things: 

1.  Take  the  Money  "Out  of  Hide" 

Whether  the  money  is  really  available  or  not,  some  agencies  "bite  the  bullet" 
and  expend  resources  originally  designated  for  other  activities  to  perform  the  work 
needed  to  justify  and  cost  a  proposed  system  or  redesign.  If  the  system  desired  is  a  new 
development,  Operation  and  Maintenance  (O&M)  funds  may  be  used— this  at  the  expense 
of  personnel  or  training.  The  House  Armed  Services  Committee  reported  that  the 
Department  of  Defense  planned  to  spend  about  nine  billion  dollars  on  ADP  systems  in 
FY90  and  that  three  quarters  of  that  amount  was  to  come  out  of  O&M  (HASC,  1  July 
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1989).  It  is  reasonable  to  assume  that  if  the  services  planned  to  spend  this  much  O&M 
to  continue  projects  already  approved,  they  would  be  willing  to  hide  the  cost  of 
requirements  analysis  and  cost  estimates  by  spending  O&M  funds  for  those  activities  as 
well. 

If  the  system  exists  but  needs  redesign,  money  may  be  diverted  from  software 
maintenance  efforts  to  plan  for  the  redesign.  In  the  case  of  RESFMS,  the  poor  initial 
design  caused  maintenance  programmers  to  be  forced  to  do  systems  analysis  and  design 
activities  just  to  adequately  perform  maintenance.  The  results  of  that  design  effort  have 
been  used  by  the  author  as  a  basis  of  cost  estimates. 

2.  Seek  Outside  Help 

Some  organizations  seek  outside  help,  either  by  paying  for  consultants,  or  by 
getting  "free"  assistance  from  other  agencies  or  sources,  such  as  Naval  Postgraduate 
School  thesis  students.  If  consultants  are  hired,  this  option  becomes  the  same  as  the 
first-funds  are  taken  "out  of  hide"  to  hire  them. 

3.  Guess 

In  order  to  meet  the  requirement  for  an  estimate  of  system  development  costs, 
some  organizations  use  a  form  of  Analogy  or  Expert  Judgement  to  provide  an  estimate. 
The  accuracy  of  the  reasoning  depends  on  many  factors  including:  historical  data 
available  from  previous  projects,  capability  and  experience  of  the  estimator,  and 
resources  that  have  been  expended  in  developing  requirements  analysis  and  design 
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proposals.  The  resulting  estimate  may  be  fairly  accurate,  or  nothing  more  than  a  wild 
guess.  For  reasons  discussed  below,  the  wild  guess  is  more  likely. 

4.  Price  to  Win 

Many  organizations  adopt  a  "Price  to  Win"  strategy  and  use  a  cost  which  is 

believed  to  be  politically  acceptable,  that  is,  a  cost  figure  that  is  judged  to  be  what  will 

be  approved.  The  practice  of  estimating  costs  based  on  external  constraints  can  also  be 

found  outside  DoD.  A  study  of  cost  estimation  methods  in  actual  use  at  Jet  Propulsion 

Laboratories  found  this  practice  prevalent  there  as  well: 

The  cost  estimating  process  can  be  seriously  impacted  by  conditions  of  severe 
budget  or  schedule  constraints.  The  result  is  that  the  estimator’s  job  becomes  less 
one  of  estimating  costs  and  more  one  of  analyzing  system  and  functional  tradeoffs. 

In  many  cases,  if  there  is  strong  motivation  to  accept  the  work,  the  job  may  be 
accepted  under  the  assumption  that  any  inconsistencies  between  requirements,  cost 
and  schedule  will  be  resolved  while  the  task  is  under  development.  ...  Thirty 
percent  of  the  respondents  reported  being  budget  constrained  while  20  percent 
reported  being  schedule  constrained.  (Hihn  and  Habib-agahi,  1991) 

If  the  politically  acceptable  approach  is  taken,  requirements  are  then 
developed  to  match  the  cost,  but  the  report  is  written  as  if  the  opposite  had  occurred. 
If  the  writers  of  the  report  understand  the  reasoning  of  reviewers  well  enough,  any 
examination  of  the  reasoning  will  find  no  problem  with  the  estimate,  for  the  requirements 
listed  and  the  cost  estimate  will  be  completely  consistent,  since  the  former  are 
constrained  by  the  latter.  Unfortunately,  the  requirements  may  not  describe  what  is 
actually  needed  by  the  organization.  So,  if  the  project  is  approved,  the  requirements  will 
undoubtedly  change  when  actual  development  begins. 
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5.  Combination 


Many  organizations  use  a  combination  of  approaches  depending  on  what 
resources  are  available,  what  the  political  climate  is  at  the  time,  and  what  expertise  is 
available. 

H.  ACCURACY  OF  DOD  COST  ESTIMATES 

How  accurate  will  these  estimates  be?  A  look  at  recent  Congressional  hearings  and 
GAO  reports  suggests  the  answer.  Reports  of  the  House  Armed  Services  Committee 
(HASC,  July  1989)(Endoso,  July  1992),  House  Appropriations  Committee  (HAC,  August 
1989),  House  Committee  on  Government  Operations  (HOC,  November  1989),  and 
numerous  GAO  reports  have  documented  serious  problems  with  DoD  information 
systems  management.  Cost  overruns  in  the  hundreds  of  millions  of  dollars  have  been 
noted.  Reasons  cited,  in  addition  to  poor  management  and  poor  communication  within 
DoD,  have  included  (HOC,  November  1989): 

•  Inaccurate  cost  and  benefit  estimates 

•  Incomplete  historical  cost  data 

•  Estimators  untrained  in  cost  estimation  techniques 

•  Excessive  requirements  growth  (changes) 

Although  the  Congressional  and  GAO  reports  did  not  make  the  connection  between 
requirements  growth  and  poor  planning  or  bad  initial  cost  estimates,  the  author  believes 
that  they  are  strongly  related.  If  the  estimate  is  flawed  to  begin  with,  those  developing 
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the  system  will  have  no  compunction  about  changing  the  requirements  later  to  suit  their 
needs.  Although  this  is  not  the  only  reason  for  requirements  growth,  it  is  a  contributing 
factor. 

I.  REASONS  FOR  POOR  DOD  COST  AND  BENEFIT  ESTIMATES 

The  reasons  for  poor  cost  and  benefit  estimates  and  the  reasons  for  cost  overruns 
and  management  failures  in  Department  of  Defense  information  system  projects  are 
interrelated.  There  is  no  one  single  cause,  and  the  several  causes  overlap  in  their  effects. 
The  reasons  for  these  problems  include  the  following: 

1.  Funds  Not  Identified  Explicitly 

Budget  managers  in  DoD  are  seeing  fewer  and  fewer  discretionary  dollars  due 
to  declining  total  budget  authority  and  the  existence  of  large  fixed-cost  operating 
expenses.  The  more  items  designated  by  Congress,  or  higher  authority,  as  "fenced"  for 
a  particular  purpose,  the  less  budget  flexibility  the  organizations  have  to  meet  their 
changing  obligations.  In  this  environment  it  is  natural  that  budget  managers  would  not 
want  funds  for  information  system  development  feasibility  studies  to  be  explicitly 
designated  in  the  budget.  Yet,  if  we  do  not  identify  these  funds  accordingly  we  will 
never  have  any  way  to  tell  how  much  we  are  really  spending  on  proposed  system  cost 
estimates.  More  important,  it  is  likely  that  little  or  no  money  will  be  spent  for  this 
purpose.  In  times  of  declining  budgets,  the  undesignated  funds  in  the  budget  will  be 
spent  for  other  contingencies.  The  question  becomes  one  of  what  we  are  willing  to  give 
up  in  order  to  provide  an  analysis  of  needs  for  the  future.  Planning  and  cost  estimation 
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activities  will  receive  scant  funding,  the  research  done  will  be  scarce,  and  the  marginal 
results  will  be  presented  as  if  a  full  and  adequate  analysis  had  been  done.  If  the  ruse  is 
not  discovered  and  the  project  is  approved,  it  is  highly  probable  that  serious  cost 
problems  will  result  due  to  extensive  changes  in  requirements  during  development. 

2.  Political  Pressure  for  a  Particular  Cost  Figure 

The  House  Committee  on  Government  Operations  identified  a  potential  cause 
of  major  system  cost  overruns  as  "the  establishment  of  arbitrary  cost  caps  by  senior 
management"  (HOC,  November  1989).  These  caps  are  usually  unrelated  to  information 
technology  (IT)  system  requirements.  The  problem  still  exists.  In  fact,  the  problem  is 
even  worse  now  in  a  period  of  dramatic  decline  in  Department  of  Defense  funding. 

3.  Poor  Historical  Data 

Good  cost  estimation  requires  good  historical  data.  Congress  identified  the 
"lack  of  credible  empirical  data  about  the  operation  to  be  automated"  (HOC,  November 
1989)  as  a  major  contributor  to  erroneous  cost  estimates  in  DoD.  Not  only  is  there  a 
lack  of  data  on  the  particular  function  to  be  automated,  but  there  is  little  data  available 
on  the  performance  of  DoD  organizations  producing  similar  projects  in  the  past. 

One  goal  of  the  Software  Engineering  Institute  (SEI)  is  to  help  organizations 
improve  the  software  development  process.  In  performing  this  function,  SEI  conducts 
Software  Process  Assessments  (SPA)  of  organizations  to  determine  their  present  level  of 
performance  and  suggestions  for  improvement.  A  significant  element  required  for 
organizations  to  progress  to  higher  levels  of  software  process  performance  is  collection 
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of  detailed  historical  and  operational  data  on  the  software  development  process  as  it  is 
performed  in  that  organization.  As  a  result  of  SPAs  conducted,  members  of  SEI  estimate 
that  most  DoD  organizations  are  functioning  at  the  lowest  level  of  Software  Process 
performance  due,  in  large  part,  to  lack  of  software  process  data  collection  and  analysis. 
(Curtis,  May  1992) 

4.  Lack  of  Trained  Estimators 

The  lack  of  individuals  trained  in  "industrial  cost  estimation  techniques  as 
applied  to  automated  systems"  (HOC,  November  1989),  identified  by  Congress  in  1989, 
is  still  a  problem.  In  response  to  this  problem,  DoD  established  the  Information 
Resources  Management  College  (IRMC)  as  part  of  the  National  Defense  University.  The 
IRMC  has  developed  a  syllabus  and  a  number  of  tools  and  resources  for  project 
managers.  Yet,  the  courses  at  IRMC  are  not  required  of  information  system  project 
managers,  nor  is  equivalent  training  required.  Worse,  the  course  has  not  been  designated 
as  officially  satisfying  training  requirements  for  Materiel  Professionals  who  will  be 
managing  software  development  projects.  Commands  wishing  to  send  personnel  to 
IRMC  must  pay  for  this  training. 

SEI  also  offers  outstanding  training  in  software  project  management,  software 
requirements  engineering,  and  numerous  other  related  skills.  DoD  personnel  may  receive 
this  training  at  considerably  lower  cost  than  individuals  from  private  industry,  yet  the 
cost  is  still  significant. 

The  training  suggested  by  Congress  and  provided  by  both  IRMC  and  SEI  is 
not  mandatory.  No  requirement  exists  for  DoD  organizations  to  have  members  who  have 


received  this  or  equivalent  training.  More  important,  budget  funding  for  training  is  often 
not  available.  Travel/training  is  one  of  the  few  discretionary  portions  remaining  in 
current  budgets.  In  times  of  budget  decline,  the  travel/training  portion  of  the  budget  is 
one  of  the  first  items  to  be  cut.  The  result  is  that,  even  if  organizations  want  to  get 
training  for  their  members,  the  funds  are  often  not  available  to  pay  for  training  fees  or 
for  the  travel  involved. 

An  added  problem  with  the  lack  of  trained  personnel  is  that,  even  when 
organizations  seek  outside  consultants  they  cannot  be  assured  of  a  good  cost  estimate. 
One  option  for  obtaining  feasibility  studies  and  cost  estimates  is  to  contract  with  another 
government  organization9,  or  some  commercial  contractor  to  provide  the  study  and 
report.  If  the  organization  contracting  for  the  study  has  no  members  trained  in  cost 
estimation  to  oversee  the  process,  there  is  no  way  to  judge  the  quality  or  applicability  of 
the  report  received. 

5.  Inadequate  Cost  Models  and  Tools 

In  order  to  produce  an  adequate  estimate,  all  the  tools  and  methods  available 
now  require  a  reasonable  estimate  of  project  size  and  complexity  as  an  input  to  the 
model.  Also  required  is  that  the  model  be  calibrated  to  a  particular  organization.  Since 
we  already  know  that,  in  DoD,  few  historical  data  are  collected,  few  tra:ned  personnel 

9An  example  of  a  government  organization  that  can  provide  needed  expertise  is  the 
Office  of  Technical  Assistance  (OTA)  of  the  U.S.  General  Services  Administration 
(GSA).  Within  OTA  are  the  Federal  Systems  Integration  and  Management  Center 
(FEDSIM)  and  the  Office  of  Software  Development  and  Information  Technology 
(OSDIT).  Both  can  provide  valuable  assistance  in  evaluating,  developing,  and  managing 
information  systems.  (GSA  Overview  Guide,  1990) 
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exist,  and  few  funds  are  available  for  requirements  analysis  prior  to  cost  estimation,  the 
models  will  produce  marginal  estimates  in  their  present  form. 

J.  THE  DOUBLE  DILEMMA 

Performing  product  design  prior  to  cost  estimation  has  disadvantages  as  well  as 
advantages.  In  a  rapidly  changing  environment,  postponing  design  allows  planners  to 
consider  many  options.  In  recent  years  rapid  technological  advances  have  been  the 
norm.  Improvements  have  been  made  in  hardware  performance  and  in  software 
languages  and  tools  available.  Postponing  detailed  design  keeps  the  project  flexible,  but 
produces  unreliable  cost  estimates.  Performing  detailed  design  prior  to  cost  estimation 
provides  much  better  cost  estimates  but  "locks  in"  a  system  architecture  that  may  be 
inadequate  or  obsolete  by  the  time  the  system  is  developed.  This  double  dilemma  is 
faced  by  all  information  system  developers,  but  it  is  most  acute  in  DoD  because  of  the 
long,  mandated  approval  and  development  process. 

Most  large  projects  for  the  Department  of  Defense  have  taken  several  years  to 
develop.  For  instance  the  House  Government  Operations  Committee  (1989)  reported  on 
four  major  DoD  systems  that  had  been  in  development  for  over  eight  years.  The  process 
of  approving  the  project  prior  to  development  is  also  complex  and  time  consuming. 
Thus,  if  a  detailed  design  is  required  as  an  input  to  a  cost  estimate,  if  a  cost  estimate  is 
required  prior  to  project  approval,  and  if  the  approval  process  takes  a  matter  of  months 
or  years,  the  whole  design  could  be  obsolete  by  the  time  development  begins.  Also, 
since  the  DoD  AIS  management  process  is  based  on  the  waterfall  model,  it  is  difficult 
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to  take  advantage  of  newer  development  models  such  as  Incremental  Development, 
Evolutionary  Prototyping,  and  Rapid  Prototyping.  If  a  new  development  model  is  used, 
documentation  to  support  the  old  waterfall  method  must  still  be  generated.  This 
documentation  requirement  increases  the  effort  required  and  extends  the  time  necessary 
for  development.  The  process  needs  to  be  changed. 

K.  RECOMMENDATIONS  TO  IMPROVE  SOFTWARE  COST  ESTIMATION 

IN  DOD 

Measures  which,  if  taken,  would  mitigate  or  eliminate  the  problems  identified  are 
as  follows: 

1.  Legitimize  Expenditures  for  Feasibility  Studies 

In  order  to  ensure  that  proper  planning  and  estimation  is  being  done,  the 
expenditure  of  funds  should  be  explicitly  authorized  for  feasibility  studies  prior  to  project 
approval.  The  approval  of  Milestone  0  and  the  Mission  Needs  Statement  should 
automatically  trigger  a  non-zero  line  item  entry  in  the  organization’s  budget  for  research 
and  development  of  the  system.  For  fully  deployed  systems  needing  redesign,  a  similar, 
but  different  milestone  system  should  be  devised  that  would  authorize  funds  to  be 
explicitly  identified  for  redesign  and  development  of  the  follow-on  system  upon  the 
approval  of  Redesign  Milestone  0.  Specific  figures  should  not  be  mandated,  but 
organizations  should  be  encouraged  to  allocate  five  percent  of  their  information  systems 
budget  to  planning  for  future  system  development.  This  authorization  should  be  included 
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in  pertinent  budget  directives  as  well  as  in  Life-Cycle  Management  directives,  so  that 
budget  managers  will  recognize  the  legitimacy  of  this  funding. 

2.  Streamline  the  Development  Process 

DoD  directives  should  be  changed  to  allow  for  new  development  models  such 
as  Incremental  Development  and  Evolutionary  Prototyping.  The  documentation 
requirements  should  be  modified  to  reflect  the  differences  in  development  models. 
Automated  tools  should  be  developed  or  acquired  to  assist  in  and  speed  up  the  design, 
evaluation,  and  development  of  systems. 

3.  Mandate  Training  and  Provide  Funding  to  Support  It 

Sufficient  numbers  of  personnel  trained  in  information  system  management 
and  cost  estimation  techniques  will  never  be  available  until  both  the  training  is  required 
and  funding  is  provided  to  obtain  it.  Congress  has  been  inclined  to  cut  DoD  budget 
authorizations  for  information  systems  when  problems  with  program  management  and 
cost  overruns  have  occurred.  Citing  unsatisfactory  progress,  Congress  recently  targeted 
$300  million  of  ADP  funding  for  possible  cuts  (Endoso,  1992).  Congress  itself 
identified  lack  of  training  as  a  problem  in  1989  (HOC,  November  1989).  Therefore, 
DoD  should  ask,  and  Congress  should  approve  the  restoration  of  targeted  ADP  funding 
with  a  stipulation  that  the  money  is  to  be  used  for  travel  and  training  expenses  for 
software  cost  estimation  and  software  management.  DoD  should  change  policies  and 
directives  to  indicate  that  IRMC  training  and  equivalent  courses  (such  as  SEI  and  Naval 
Postgraduate  School)  are  allowed  and  required  of  all  software  project  managers  and  their 
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assistants.  The  goal  should  be  to  ensure  that  trained  personnel  will  be  present,  by  the 
end  of  FY93,  in  every  DoD  organization  that  performs  development. 

4.  Mandate  Historical  Data  Collection  and  Set  Standards 

Collection  of  historical  data  on  software  development  efforts  is  essential  and 
should  be  mandatory  for  not  only  DoD  agencies  but  also  all  contractors  that  work  for 
DoD  agencies.  The  data  collected  should  be  available  for  reference  at  the  local 
command  where  it  was  gathered  and  as  part  of  a  central  DoD  repository.  Contractor 
data  should  be  supplied  to  the  database  as  part  of  contract  deliverables.  In  order  for  this 
to  be  effective  and  not  be  onerous,  DoD  should  not  only  stipulate  required  data  format 
but  also  provide  automated  tools  to  aid  in  collecting  the  data.  Ideally,  historical  data 
collection  would  be  built  into  and  integrated  with  project  management  tools.  This  would 
both  increase  the  chances  that  the  data  would  be  collected  and  eventually  improve  the 
project  management  process. 

5.  Improve  the  Cost  Models 

Although  implementing  the  previous  four  recommendations  would 
dramatically  improve  the  accuracy  of  current  cost  models,  research  should  continue  into 
improved  costing  models  and  tools.  Several  DoD  organizations  are  attempting  currently 
to  provide  better  costing  tools.  SASET  contains  an  example  of  an  attempt  to  perform 
cost  estimation  in  a  new  way.  Developed  by  Martin  Marietta  Denver  Astronautics 
division  under  contract  to  the  Air  Force  Cost  Center,  SASET  is  a  proprietary  algorithmic 
model  for  cost  estimation  that  operates  in  two  modes.  If  size  of  the  project  in  LOC  is 
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available  a  cost  estimate  can  be  produced  using  conventional  means.  SASET  is 
innovative,  however,  in  that  it  has  a  mode  in  which  a  description  of  the  function  to  be 
performed  can  be  used  to  generate  an  estimate  of  effort  required.  The  input  required  is 
not  a  function  point  analysis  as  described  by  Albrecht,  but  a  description  of  module 
function  such  as:  "Accounts  Receivable  Accounting  Module."  Based  on  a  database  of 
prior  DoD  projects,  an  average  size  and  effort  required  to  program  a  module  performing 
that  function  is  estimated.  This  is  definitely  a  move  in  the  right  direction,  but  it  needs 
to  be  refined.  The  database  of  DoD  projects  is  too  small  and  contains  mostly  real-time 
system  development  statistics.  The  model  may  be  calibrated  to  a  particular  site,  but  that 
process  presents  the  same  problems  as  other  cost  models. 

If  innovative  cost  estimation  models  can  be  developed  to  be  integrated  with 
the  project  management  tools  suggested  above  that  also  collect  historical  data,  significant 
progress  can  be  made  toward  achieving  both  accurate  cost  estimates  and  proficient 
program  management. 
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VI.  COST  AND  LIFETIME  ESTIMATES 


A.  COSTS  CONSIDERED  FOR  RESFMS 

Two  alternatives  for  RESFMS  will  be  compared,  a  status  quo  alternative  and  a 
redesign  alternative.  Costs  for  the  status  quo  alternative  were  determined  by  examining 
budget  documents  for  RESFMS  and  through  interviews  with  Mr.  Ron  Chamberlain, 
COMNAVRESFOR  Budget  Director  (13  May  1992)  and  LCDR  Furrey,  RESFMS 
Redesign  Project  Manager  (14  May  1992).  To  determine  costs  for  the  redesign 
alternative,  cost  estimation  techniques  had  to  be  applied  to  data  provided  by 
COMNAVRESFOR  staff  and  SEMA  contractor  personnel. 

1.  Costs  for  Status  Quo  Alternative 

The  largest  single  cost  item  for  RESFMS  in  its  current  configuration  is 
payment  for  services  provided  by  NCTS  New  Orleans.  NCTS  provides  both  hardware 
operation  and  service  to  the  Naval  Reserve.  In  addition  to  operating  the  Unisys  1 100/92 
mainframe  computer,  associated  Input/Output  devices,  and  mass  storage,  NCTS  also 
provides  four  individuals  to  operate  a  help  desk.  The  help  desk  accepts  calls  from  users 
seeking  information  and  assistance.  The  help  desk  personnel  answer  questions  and  refer 
problems  to  technical  staff  for  corrective  action. 

NCTS  New  Orleans  is  a  Navy  Industrial  Fund  activity.  Such  activities 
receive  their  initial  funding  from  a  revolving  fund  called  the  Defense  Business  Operations 
Fund  (DBOF),  formerly  referred  to  as  the  Navy  Industrial  Fund  (NIF).  Because  the 
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term  DBOF  is  relatively  new,  most  Navy  personnel  still  refer  to  such  activities  as  NIF 
activities.  NIF  activities  perform  services  for  other  DoN  or  DoD  activities  and  charge 
those  customer  activities  according  to  the  cost  of  providing  the  service.  The  customer 
activities  pay  for  NIF  services  out  of  their  appropriated  funds  budget.  (Practical 
Comptrollership,  January  1992)  These  monies  are  commonly  referred  to  as  "NIF 
charges." 

NIF  charges  for  RESFMS  operation  were  $3,040,000  in  FY90  and 
$3,514,000  in  FY91  (CNRF  Ad  Hoc  Budget  Expenditure,  1992).  With  optimistic 
estimates,  the  lowest  projected  figure  for  NIF  charges  in  the  next  6  years  is  $2,841,000 
in  FY93  (CNRF  Ad  Hoc  Budget  Projection,  1992).  From  discussions  with  LCDR 
Furrey  (14  May  1992)  and  Mr.  Chamberlain  (13  May  1992),  I  believe  that  prudent 
management  may  hold  NIF  charges  between  the  FY90  and  FY91  figures.  Although  the 
FY93  figure  is  possible,  it  is  not  probable  that  costs  can  be  reduced  to  that  degree. 
Inadequate  data  exist  to  determine  a  distribution  of  values  other  than  these  three  points. 
Rough  modeling  can  still  be  done,  however,  using  a  triangular  distribution  (Mendelsohn 
et  al.,  1991).  Therefore,  using  a  triangular  distribution,  I  have  selected  $3,514,000  as 
the  high  value,  $2,841,000  as  the  low  value  and  $3,040,000  as  the  most  probable, 
producing  an  expected  value  of  $3,131,667  for  NIF  costs. 

The  second  largest  cost  factor  in  the  current  configuration  is  payment  to 
contractors  for  software  maintenance.  In  FY91,  COMNAVRESFOR  paid  $1,608,000 
to  SEMA,  the  contractor  currently  providing  software  maintenance  for  RESFMS.  In 
FY92,  because  of  DoD  budget  cuts,  the  software  maintenance  budget  has  been  capped 
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at  $1,050,000.  Future  DoD  budget  cuts  may  force  further  reductions  in  the  RESFMS 
maintenance  budget.  The  absolute  minimum  acceptable  maintenance  level  for  RESFMS 
in  its  current  configuration  is  nine  full  time  maintenance  programmers  (Furrey,  1992). 
COMNAVRESFOR  pays  SEMA  $4,853  per  man-month  for  services  rendered.  This 
figure  includes  all  overhead  and  fringe  benefits  (Chamberlain,  1992).  Nine  programmers 
for  one  year  at  this  rate  would  cost  $524,124  and  this  is,  therefore,  the  minimum 
acceptable  maintenance  budget.  Using  a  triangular  distribution  with  $1,608,000  as  the 
high,  $524, 124  as  the  low  and  the  current  $1,050,000  as  the  most  likely  figure,  produced 
an  expected  value  for  software  maintenance  of  $1,116,393. 

On  COMNAVRESFOR  staff  are  five  government  civilian  employees,  two 
managers  and  three  support  personnel,  who  are  assigned  to  RESFMS.  Total  cost  for 
civilian  personnel,  including  fringe  benefits,  is  $307,000  (CNRF  Ad  Hoc  Budget 
Execution,  1992).  Expenses  for  supplies,  including  software  provided  to  contractors, 
diskettes  distributed  to  field  sites,  and  office  supplies  was  $100,000  in  FY91  (CNRF  Ad 
Hoc  Budget  Execution,  1992).  Cost  to  the  Naval  Reserve  of  operating  computer  and 
support  equipment  at  COMNAVRESFOR  headquarters  in  support  of  RESFMS  was 
$60,000  (CNRF  Ad  Hoc  Budget  Execution,  1992).  RESFMS  management  and  support 
personnel  are  required  to  travel  in  order  to  provide  training  for  users  and  to  attend 
management  meetings  in  Washington,  DC.  Travel  associated  with  RESFMS  cost 
$34,000  in  FY91  (CNRF  Ad  Hod  Budget  Execution,  1992).  I  believe  these  figures  are 
representative  of  what  can  be  expected  in  future  years.  Expected  annual  costs  for  the 
Status  Quo  Alternative  which  will  be  used  in  this  analysis  are  summarized  in  Table  III. 
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Table  III  RESFMS  STATUS  QUO  COSTS 


NIF  Charges  to  NCTS 

$3,131,667 

Software  Maintenance  (SEMA) 

$1,116,393 

CNRF  Civilian  Personnel 

$307,000 

Supplies 

$100,000 

Operations  at  CNRF 

$60,000 

Travel 

$34,000 

Annual  Costs 

$4,749,060 

2.  Costs  for  Redesign  Alternative 

a.  Software  Cost  Considerations 

The  largest  single  cost  item  for  the  redesign  of  RESFMS  is  the  cost  of 
software  development.  After  system  implementation,  the  largest  cost  in  any  single  year 
will  be  software  maintenance  expense.  In  order  to  determine  the  amount  of  these  costs, 
software  cost  estimation  methods,  such  as  those  discussed  in  Chapter  V,  must  be  used. 
To  determine  the  cost  of  software  development,  an  estimate  of  effort  in  terms  of  man- 
months  is  calculated.  The  number  of  man-months  required  is  then  multiplied  by  the 
known  cost  of  a  contractor  man-month  to  produce  a  dollar  cost  estimate. 

Also  important  to  economic  analysis  is  the  length  of  time  required  for 
the  development  effort,  or  the  number  of  schedule  months,  since  this  figure  affects  the 
amount  of  lead  time  required.  Nine  man-months  of  effort  can  be  accomplished  in  nine 
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months  by  one  programmer,  or  in  one  month  by  nine  programmers.  Distribution  of 
effort  in  a  software  project  has  been  shown  to  take  on  the  shape  of  a  classic  curve  first 
described  analytically  by  Lord  Rayleigh  (Pressman,  1992).  Norden  (1980)  analyzed 
empirical  data  from  software  development  projects  to  confirm  that  the  Rayleigh  curve 
describes  the  optimum  distribution  of  effort.  Many  software  cost  estimation  tools 
incorporate  Rayleigh  curve  calculations  of  distribution  to  determine  an  optimum  schedule 
for  development  of  the  project  being  estimated. 

b.  COMNAVRESFOR  Estimation  of  Development  Effort 

When  the  redesign  of  RESFMS  was  first  proposed,  in  mid  1991, 
COMNAVRESFOR  management  personnel  asked  SEMA  contractors  for  an  estimate  of 
effort  and  schedule  required  to  reprogram  the  system.  The  requested  estimate  was 
constrained  by  the  assumption  that  only  nine  programmers  would  be  used  for 
development  and  that  this  number  would  remain  constant  throughout  development.  The 
target  configuration  for  RESFMS  was  also  not  fully  determined  at  the  time  of  this 
estimate.  Based  on  expert  opinion  of  maintenance  programmers,  analogy  with  similar 
projects  performed  by  other  SEMA  programmers,  and  the  constraints  given,  an  estimate 
of  77  schedule  months  for  nine  programmers,  or  a  total  of  693  man-months  of  effort  was 
produced  (CNR F  RESFMS  Briefing,  August  1991).  In  June  of  1992,  after  target  system 
configuration  had  been  decided,  programming  language  and  tools  chosen,  and  initial 
system  design  nearly  complete,  the  estimate  was  revised  to  42  schedule  months  for  nine 
programmers,  or  a  total  of  378  man-months  of  effort. 
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For  purposes  of  this  analysis,  I  chose  to  derive  an  estimate  of 
development  effort  and  cost  by  independent  means  from  those  used  by 
COMNAVRESFOR  staff  personnel.  Using  sizing  and  functional  data  provided  by 
COMNAVRESFOR  and  SEMA  personnel,  I  have  produced  estimates  with  three  different 
cost  estimation  tools. 

c.  Function  Point  Estimate 

Two  of  the  18  software  maintenance  programmers  and  analysts  for 
RESFMS  have  been  tasked  with  preparing  for  RESFMS  redesign.  To  date,  system 
requirements,  preliminary  design,  and  product  design  are  essentially  complete  (Furrey, 
14  May  1992).  Using  this  data,  SEMA  personnel  have  calculated  function  points  (FP) 
for  the  redesigned  system  using  procedures  devised  by  Albrecht  (1984)  and  a  program 
for  IBM  compatible  personal  computers  written  in  Pascal  by  SEMA  programmers.  The 
result  of  their  analysis  for  each  of  the  three  major  modules  of  RESFMS  is  shown  in 
Table  IV  (Lazar,  16  July  1992). 

Table  IV  UNADJUSTED  FUNCTION  POINT  COUNT 

RESFMS  Module  Unadjusted  FP 

AT/ADT  763 

Travel  655 

RPN  Accounting  1252 

Total  System  2670 
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If  we  assume  that  relative  functionality  should  be  an  indicator  of  relative 
program  size,  examination  of  the  above  data  indicates  that  AT/ADT  should  be 
approximately  61%  and  Travel  52%  as  large  as  RPN  Accounting. 

To  obtain  the  final  function  point  value,  the  initial  function  point  count 
(FC)  is  adjusted  for  general  characteristics  (GC).  There  are  14  general  characteristics 
and  each  may  have  a  degree  of  influence  (DI)  value  between  zero  and  five.  DI  is 
subjectively  determined  for  each  of  the  14  general  characteristics  and  these  values  are 
summed  to  produce  a  single  GC  value.  The  final  function  point  measure  (FP)  is 
determined  by  the  following  formula  (Albrecht,  1984): 

FP  =  FC  [0.65  +  (0.01  X  GC)  ] 

SEMA  analysts  determined  that  GC  for  the  redesigned  RESFMS  should 
be  39  (Lazar,  16  July  1992).  Using  this  value  in  the  formula  above  produces  a  function 
point  value  of  2776.8  for  the  redesigned  system. 

Since  determination  of  degree  of  influence  for  each  of  the  general 
characteristics  is  subjective  and  descriptions  given  allow  ranges  of  values  (Albrecht, 
1984),  different  values  for  GC  could  be  reasonably  determined.  I  have  gained  knowledge 
of  RESFMS  system  characteristics  through  my  use  of  the  current  system  in  my  previous 
two  commands.  Further,  I  have  learned  how  the  system  will  change  in  the  redesign 
through  interviews  with  Lacy  (30  June  1992),  Furrey  (14  May  1992),  and  Lazar  (16  July 
1992).  Using  this  knowledge,  I  examined  the  descriptions  of  function  point  general 
characteristics  (Albrecht,  1984)  and  calculated  the  possible  GC  minimum  and  maximum 
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values.  Minimum  GC  was  calculated  by  taking  known  characteristics  of  RESFMS  and 
assigning  the  minimum  suggested  numeric  value  for  degree  of  influence  for  each  of  those 
characteristics.  Using  this  methodology  a  minimum  GC  of  21  was  obtained.  Repeating 
the  process,  but  taking  the  maximum  value  suggested  in  each  case,  a  maximum  GC  of 
55  was  obtained.  Using  the  minimum  GC  of  21  and  unadjusted  function  point  count 
provided  by  SEMA  analysts,  a  function  point  value  of  2296.2  is  obtained  for  RESFMS. 
If  the  maximum  GC  of  55  is  used,  a  value  of  3204.0  is  the  result. 

If  function  points  have  been  used  at  a  software  development  activity 
consistently  and  historical  data  has  been  collected  and  retained  for  previous  projects,  a 
measure  of  effort  required  can  be  computed  in  terms  of  function  points  per  man-month. 
Unfortunately,  neither  SEMA  nor  COMNAVRESFOR  have  collected  and  retained  such 
consistent  data.  However,  an  estimate  of  effort  to  produce  this  system  can  still  be  made, 
but  it  will  have  to  rely  on  rules  of  thumb  for  general  industry  averages  and  thus  be  less 
accurate  than  that  derived  from  empirical  data.  To  compensate  for  the  lack  of  accuracy, 
I  made  several  estimates  in  order  to  determine  a  feasible  range  for  the  desired  value. 

Industry  averages  for  productivity  in  terms  of  function  points  tend  to 
fall  between  Five  and  six  function  points  per  man-month  of  effort  (Pressman,  1992; 
Zwieg,  9  May  1992).  Use  of  fourth-generation  languages  or  integrated  Computer  Aided 
Software  Engineering  (CASE)  tools  can  increase  productivity  to  15  or  20  function  points 
per  man-month  (FP/MM).  Programmers  recoding  RESFMS  will  be  using  an  integrated 
CASE  environment  called  Ada  Sage,  which  was  produced  under  contract  to  the  U.S. 
Department  of  Energy  and  is  available  at  no  cost  to  government  agencies.  The  output 
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of  Ada  Sage  is  Ada  programming  code  which  is  then  compiled  for  the  target  hardware 
configuration.  Even  though  integrated  CASE  tools  increase  productivity,  manual  coding 
is  sometimes  required  to  adapt  the  output  to  a  particular  system  configuration  or  function. 
Also,  any  new  programming  tool  set  or  environment,  no  matter  how  easy  to  use,  requires 
some  amount  of  time  for  programmers  and  analysts  to  acquaint  themselves  with  the 
features  of  the  tool  and  become  adept  at  its  use  (Zwieg,  9  May  1992). 

The  RESFMS  redesign  project  will  be  the  first  time  that  SEMA 
programmers  have  used  Ada  Sage  and  the  first  time  they  have  programmed  in  the  Ada 
language.  The  Navy  Center  for  Cost  Analysis  has  determined  that  organizations  using 
Ada  experience  significant  savings  in  software  maintenance  effort  and  expense  as 
compared  to  those  using  other  programming  languages.  Savings  are  also  apparent  in 
initial  development  efforts,  for  second  and  subsequent  projects.  However,  a  large 
penalty,  in  the  form  of  training  time,  is  imposed  on  organizations  in  their  first  use  of  Ada 
(Gross,  5  May  1992).  Therefore,  productivity  gains  associated  with  use  of  CASE  tools 
will  probably  be  negated  by  loss  of  productivity  due  to  training  time  required  for  the  first 
use  of  Ada.  Considering  all  the  above,  I  have  used  both  the  five  FP/MM  and  six 
FP/MM  productivity  figures  and  the  low,  high,  and  SEMA  estimates  of  GC  to  produce 
six  possible  values  for  RESFMS  redesign  effort  using  function  points.  These  estimates 
are  shown  in  Table  V. 
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Table  V  TOTAL  MM  EFFORT  BASED  ON  FUNCTION  POINT  ESTIMATES 


FP  Value 

5  FP/MM 

6  FP/MM 

Low  GC  Value 

2296.2 

459.24 

382.70 

High  GC  Value 

3204.0 

640.80 

534.00 

SEMA  Estimate 

2776.8 

555.36 

462.80 

d.  Estimate  of  Redesign  Effort  Using  LOC 

As  discussed  previously,  a  number  of  models  exist  which  use  lines  of 
code  (LOC)  as  a  size  input  to  produce  an  estimate  of  effort  required  for  software 
development.  Two  factors  significantly  affect  the  accuracy  of  a  particular  model’s 
estimate:  whether  or  not  the  model  has  been  calibrated  for  the  particular  organization 
performing  the  development  and  the  accuracy  of  the  LOC  input  estimate. 

Insufficient  historical  data  exists  to  calibrate  models  specifically  to 
Commander  Naval  Reserve  Force  or  SEMA  projects.  However,  the  problem  of 
calibration  could  be  mitigated  if  models  were  used  which  had  been  calibrated  to 
Department  of  the  Navy  or  Department  of  Defense  projects.  Models  so  calibrated  would 
be  less  accurate  in  this  analysis  than  if  they  had  been  calibrated  for  the  specific 
development  organization,  but  considerably  more  accurate  than  models  calibrated  for 
civilian  industry  use. 

In  the  case  of  RESFMS,  a  status  quo  system  exists  which  may  be  used 
as  a  basis  for  LOC  estimates  of  the  redesign.  In  June  1992  a  utility  program  was  run 
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by  SEMA  maintenance  personnel  which  counts  lines  of  code  in  COBOL.  This  program 

was  used  to  determine  the  exact  LOC  counts  for  each  RESFMS  module.  The  results  as 

n  ported  by  Ben  Lazar  (16  July  1992)  were: 

Table  VI  RESFMS  CURRENT  CONFIGURATION  LINES 
OF  CODE 


Program  Module 

LOC 

Total  LOC 

RPN 

275,668 

RPN  PROC 

60,831 

Total  RPN 

336,449 

AT/ADT 

572,532 

Travel 

361,227 

AT/ADT-Travel  PROC 

89,190 

Total  AT/ADT-Travel 

1,022,949 

Total  RESFMS 

1,359,398 

The  term  PROC  in  the  above  table  refers  to  modules  of  program  code 
that  are  common  support  modules  called,  and  used  repeatedly,  by  applications  programs. 
Thus  RPN  PROC  is  the  LOC  for  support  programs  used  by  the  RPN  accounting 
application  module,  and  AT/ADT-Travel  PROC  is  a  LOC  count  of  support  programs 
used  in  common  by  both  the  AT/ADT  and  the  Travel  modules.  In  order  to  compute  the 
total  LOC  for  each  subsystem,  the  RPN  PROC  LOC  should  be  added  to  RPN  and  the 
AT/ADT-Travel  PROC  LOC  should  be  split  evenly  between  AT/ADT  and  Travel  with 
half  added  to  each  (Lazar,  16  July  1992). 
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Special  note  should  be  taken  of  the  relative  size  of  the  modules.  From 
function  point  analysis  it  was  determined  that  the  AT/ADT  module  should,  ideally,  be 
61%  as  large  as  RPN  Accounting,  and  that  Travel  should  be  52%  as  large  as  RPN.  If 
it  is  assumed  that  the  RPN  PROC  is  counted  as  part  of  the  RPN  function  and  that  the 
AT/ADT-Travel  PROC  may  be  equally  divided  among  the  two  modules  which  share  the 
PROCs,  then  RPN  Accounting  is  336,449  LOC,  AT/ADT  is  617,127  LOC,  and  Travel 
is  405,822  LOC.  Thus,  AT/ADT  is  actually  183%  and  Travel  121%  as  large  as  RPN 
Accounting.  Why  is  there  such  a  disparity  between  the  actual  figures  and  those  one 
would  predict  based  on  function  points? 

The  answer  lies  in  the  way  RESFMS  was  developed.  The  AT/ADT 
module  and  the  Travel  module  were  the  first  programmed.  They  were  also  both 
produced  by  NARDAC  New  Orleans.  RPN  Accounting  was  produced  by  civilian 
contractors.  It  is  obvious  from  both  historical  data  (Lacy,  30  June  1992)  and  current 
condition  of  the  code  (Lazar,  16  July  1992)  that  little  or  no  design  was  done  prior  to  the 
start  of  coding,  that  structured  programming  techniques  were  not  used,  and  that  huge 
amounts  of  code  was  redundantly  copied  throughout  the  modules.  The  AT/ADT  and 
Travel  modules  are,  therefore,  considerably  larger  than  optimum  design  would  indicate. 
On  the  other  hand,  SEMA  maintenance  programmers  assert  that  the  RPN  Accounting 
module  is  well  designed  and  reasonably  efficient  (Lazar,  16  July  1992).  RPN 
Accounting  is  probably  10%  larger  than  optimum  due  to  growth  from  software 
maintenance  patches  inserted  during  the  last  seven  and  a  half  years  of  operation  (Lazar, 
16  July  1992),  but  is  otherwise  well  programmed.  Therefore,  consensus  of  RESFMS 
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managers  and  programmers  (Furrey,  14  May  1992)(Lacy,  30  June  1992)(Lazar,  16  July 
1992)  is  that  the  proportions  calculated  from  function  point  analysis  are  very  close  to 
optimum  for  a  redesigned  system  and  the  RPN  Accounting  module  may  be  used  as  a 
baseline  to  determine  what  size  the  programs  should  be  if  written  in  COBOL. 

Using  this  reasoning  I  have  developed  both  a  Low  and  a  High  estimate 
for  LOC  to  be  used  as  input  for  cost  estimation  models.  Both  estimates  assume  that  the 
relative  size  of  the  RPN  PROC  to  RPN  Accounting  module  is  consistent  with  the  amount 
of  support  programs  to  applications  programs  that  will  be  present  in  the  final  system. 
The  High  estimate  assumes  that  the  RPN  Accounting  module  is  the  correct  size,  as  is, 
for  the  redesigned  system  and  makes  no  adjustment  to  LOC  estimate  for  RPN 
Accounting.  In  the  High  estimate,  AT/ADT  is  computed  at  61%  of  the  size  of  RPN 
Accounting  and  Travel  is  computed  as  52%  of  RPN  Accounting. 

The  Low  estimate  assumes  that  RPN  Accounting  is  10%  too  large  at 
present  and  adjusts  its  size  downward.  Also,  consideration  is  given  for  the  fact  that  the 
target  system  will  be  programmed  in  Ada  which  is  more  efficient  than  COBOL.  Recall 
that,  on  average,  Ada  requires  only  70%  as  many  lines  of  code  as  COBOL  to  produce 
the  same  relative  functionality  (Pressman,  1992).  Therefore,  I  have  further  adjusted  the 
RPN  Accounting  module  downward  to  account  for  the  use  of  Ada.  If  we  adjust  90%  for 
maintenance  growth  and  70%  for  Ada  use,  our  final  estimate  will  be  63%  as  large  as  the 
original  (0.9  x  0.7  =  0.63).  The  result  is  that  the  estimate  for  RPN  Accounting  is 
reduced  from  its  present  size  of  336,500  LOC  in  COBOL  to  approximately  212,000  LOC 
in  Ada.  The  Low  estimate  takes  this  as  the  new  baseline  for  RPN  Accounting  and 
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calculates  AT/ADT  as  61%  and  Travel  as  52%  of  this  reduced  baseline.  A  summary  of 
LOC  estimates  used  for  this  analysis  is  found  in  Table  VII  below. 

Table  VII  LOC  VALUES  USED  FOR  RESFMS  COST  ESTIMATES 


Module 

Low  LOC  Estimate 

High  LOC  Estimate 

RPN 

173,850 

276,000 

RPN  Support 

38,150 

60,500 

RPN  Total 

212,000 

336,500 

AT/ADT 

106,200 

168,250 

AT/ADT  Support 

23,300 

37,000 

AT/ADT  Total 

129,500 

205,250 

Travel 

90,600 

143,500 

Travel  Support 

19,900 

31,500 

Travel  Total 

110,500 

175,000 

RESFMS  Total 

452,000 

716,750 

A  third  LOC  estimate  to  be  considered  is  a  variation  of  the  Low  LOC 
estimate.  One  of  the  goals  of  the  CIM  initiative  is  increased  reuse  of  software  within 
DoD.  The  Director  of  Defense  Information  has  stated  that  one  goal  of  CIM  is  to  achieve 
50%  reuse  of  code  in  developing  new  systems.  The  RPN  Accounting  module  of 
RESFMS  is  a  large  financial  accounting  program  with  accommodations  to  the  unique 
aspects  of  Naval  Reserve  record  keeping.  It  is  very  likely  that  up  to  50%  of  the  RPN 
Accounting  module  could  be  provided  by  reused  generic  accounting  modules.  This 
possibility  would  greatly  reduce  the  effort  required  to  redesign  the  entire  RESFMS 
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system.  To  allow  for  the  possibility  of  incorporating  reused  Ada  programs  in  RESFMS, 
I  have  added  a  third  LOC  estimate  which  adjusts  the  Low  LOC  estimate  to  account  for 
50%  of  the  RPN  Accounting  module  being  constructed  of  reused  code. 

The  Naval  Center  for  Cost  Analysis,  in  Washington,  DC,  distributes 
two  software  cost  estimation  tools  that  use  LOC  as  an  input:  REVIC,  for  REVised 
Intermediate  Cocomo,  and  SASET,  for  Software  Architecture,  Sizing,  and  Estimating 
Tool.  Both  are  available  in  versions  which  run  on  personal  computers.  Using  these  two 
cost  estimating  tools  and  the  three  LOC  estimates  described  above,  I  have  produced  a 
series  of  estimates  of  effort  required  for  development  and  for  maintenance  effort 
necessary  to  sustain  a  redesigned  RESFMS. 

e.  REVIC  Estimate  of  RESFMS  Redesign  Effort 

REVIC  is  simply  the  Intermediate  COCOMO  model  (Boehm,  1981) 
adapted  for  use  in  the  Department  of  Defense.  It  is  revised  because  it  has  been  calibrated 
based  on  empirical  data  from  DoD  projects,  and  because  it  adds  a  new  development 
mode,  the  Ada  mode,  to  account  for  the  difficulties  of  using  Ada  for  the  first  time 
(Gross,  5  May  1992).  The  user  is  queried  for  values  for  20  Environmental  Factors,  such 
as  programmer  capability  and  database  size,  that  are  used  to  compute  the  environmental 
modifier  in  the  COCOMO  model  (Kile,  1991).  Just  as  COCOMO  allows  for  adjustments 
to  its  calculation  for  software  modification  efforts,  REVIC  queries  the  user  to  determine 
if  a  particular  module  is  being  produced  as  an  initial  development  effort  or  as  a 
modification  of  existing  code.  If  modules  are  designated  as  being  modified,  REVIC  asks 
what  percent  of  redesign  and  recode  will  be  required.  The  result  of  REVIC’s 
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calculations  is  an  estimate  of  effort  required,  in  man-months,  and  an  estimate  of  the 
optimum  number  of  schedule  months.  These  estimates  are  displayed  for  each  phase  of 
development  as  well  as  a  summary  of  the  total  effort.  REVIC  also  computes  an  estimate 
of  life  cycle  maintenance  effort  required  for  15  years  of  operation. 

REVIC  divides  effort  and  schedule  estimates  into  those  required  for  six 
phases  of  system  development: 

•  Software  Requirements  Engineering 

•  Preliminary  Design 

•  Critical  Design 

•  Code  &  Unit  Testing 

•  Integration  &  Test 

•  Development  Test  &  Evaluation 

Software  analysts  working  on  RESFMS  have  completed  activities 
equivalent  to  the  first  two  phases  of  the  REVIC  model,  that  is  Software  Requirements 
Engineering  and  Preliminary  Design.  Therefore,  estimates  used  for  this  analysis  and 
derived  using  REVIC  have  had  the  values  for  Software  Requirements  Engineering  and 
Preliminary  Design  phases  subtracted  from  the  totals  reported  by  REVIC. 

The  redesign  of  RESFMS  will  largely  involve  recoding  existing  COBOL 
programs  in  a  new  language,  Ada.  Actual  change  in  functionality  will  be  small.  SEMA 
analysts  estimate  that  functional  design  will  change  only  10%  to  30%  from  current 
functionality  (Lazar,  16  July  1992).  Even  in  the  AT/ADT  and  Travel  modules  that  are 
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so  poorly  coded,  the  actual  design  will  change  little.  Almost  all  the  gains  in  efficiency 
will  result  from  improved  coding  practice  and  use  of  common  modules  instead  of 
redundant  code.  Even  though,  in  this  analysis,  the  values  computed  for  the  Preliminary 
Design  Phase  will  be  discarded,  it  is  important  to  accurately  estimate  the  amount  of 
design  change  in  a  COCOMO  software  modification  cost  estimate.  This  is  because  the 
design  activity  is  not  confined  to  those  phases  with  "design"  in  their  title  (Boehm,  1981). 
Design  activities  occur  in  varying  amounts  in  almost  all  phases.  Thus,  the  percent 
change  in  design  affects  estimates  of  effort  for  all  phases  of  development.  For  this 
evaluation  I  have  chosen  a  design  change  value  of  20%  for  use  in  all  REVIC  cost 
estimations. 

Results  of  REVIC  calculations  using  the  three  described  LOC  estimates 
as  input  and  subtracting  values  for  Software  Requirements  Engineering  and  Preliminary 
Design  phases  were  as  follows: 


Table  VIII  REVIC  DERIVED  ESTIMATES 


LOC  Estimate 

Total  MM 

Sched  Mos 

50%  Reuse  in  RPN 

321.3 

25.9 

Low  LOC  Estimate 

356.1 

26.7 

High  LOC  Estimate 

504.9 

29.8 

/.  SASET  Estimate  of  RES FMS  Redesign  Effort 

SASET  was  developed  by  Martin  Marietta  Denver  Aerospace 
Corporation  under  contract  to  the  Air  Force  Cost  Center.  The  Naval  Center  for  Cost 
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Analysis  has  validated  SASET  for  Navy  projects  using  empirical  data  from  25  DoD 
software  development  projects  (Gross,  5  May  1992).  SASET  uses  proprietary  algorithms 
for  its  calculations.  An  interesting  feature  is  that  LOC  estimates  can,  and  should,  be 
divided  up  into  systems  programs,  application  programs,  and  support  programs  if 
possible.  Also,  whether  the  code  is  being  initially  programmed,  modified,  or  ported  to 
a  new  hardware  platform  makes  a  difference  in  estimates  of  effort  required  to  develop. 
The  more  specific  the  user  can  be  concerning  the  function,  use,  and  development  pattern 
of  individual  modules,  the  better  will  be  the  estimate  that  SASET  produces  (Silver  et  al., 
1990).  In  addition  to  a  LOC  estimate,  the  user  is  required  to  enter  values  for  32  factors 
concerning  system  characteristics,  programmer  proficiency,  and  development 
environment.  The  values  for  these  32  factors  are  used  to  determine  both  budget  and 
schedule  multipliers  that  affect  the  estimate  of  effort  and  schedule  required.  Output  may 
be  obtained  in  a  number  of  forms.  I  chose  to  receive  a  measure  of  effort  in  man-months 
and  optimum  schedule  in  calendar  months  by  phase  of  development.  Maintenance  effort 
required  can  be  computed  if  the  user  so  desires 

The  description  of  RESFMS  PROC  module  functions  (Lazar,  16  July 
1992)  is  close  to,  but  does  not  exactly  match,  the  description  of  support  software  in 
SASET  estimation  (Silver,  1990).  To  provide  for  the  possibility  of  error  in  designation 
of  software  type,  I  have  used  SASET  to  calculate  estimates  both  with  support  software 
(PROC  modules)  separate  and  with  them  lumped  into  the  application  LOC  estimate. 
RESFMS  redesign  will  not  involve  coding  any  systems  software.  In  this  case  then, 
entries  in  SASET  would  be  appropriate  only  for  applications  and  support  module  LOC. 
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Since  breaking  out  support  software  from  applications  software  LOC  involves  two  entries 
each  for  RPN,  AT/ADT,  and  Travel  modules,  I  refer  to  this  entry  format  as  ”6  Module.” 
Since  lumping  PROC  modules  with  its  associated  application  involves  only  one  entry 
each  for  the  three  major  modules,  I  refer  to  this  option  as  "3  Module." 

Software  development  as  defined  by  SASET  is  broken  into  ten  phases: 

•  Systems  Requirements 

•  Requirements  Allocation 

•  Software  Requirements 

•  Preliminary  Design 

•  Detailed  Design 

•  Code 

•  Checkout 

•  Unit  Testing 

•  Physical  and  Formal  Quality  Testing,  Integration 

•  Systems  Test  &  Integration 

SEMA  software  analysts  working  on  RESFMS  redesign  have  already  completed  actions 
equivalent  to  the  first  four  phases  computed  by  SASET.  Therefore,  estimates  used  for 
this  analysis  and  derived  using  SASET  have  had  the  values  for  Systems  Requirements, 
Requirements  Allocation,  Software  Requirements,  and  Preliminary  Design  subtracted 
from  the  totals  reported  by  SASET. 
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In  order  to  produce  a  reasonable  range  of  estimates  using  SASET,  five 
sets  of  input  values  were  used:  Low  LOC  with  50%  reuse  of  code  in  the  RPN  module 
(called  "50%  Reused"  in  the  tables  and  graphs;,  Low  LOC  3  Module,  Low  LOC  6 
Module,  High  LOC  3  Module,  and  High  LOC  6  Module.  The  results  of  these 
calculations  are  shown  in  Table  IX. 

Table  IX  SASET  DERIVED  ESTIMATES 


LOC  Estimate 

Total  MM 

Sched  Mos 

50%  Reused 

336.79 

20 

Low  LOC  6  Module 

439.96 

21 

Low  LOC  3  Module 

488.82 

21 

High  LOC  6  Module 

697.71 

25 

High  LOC  3  Module 

775.15 

25 

g.  Summary  of  Software  Development  Effort  Estimates 

Using  the  three  cost  estimation  tools,  a  total  of  14  estimates  of  effort 
were  derived:  three  using  Function  Point  Analysis,  five  using  REVIC,  and  six  using 
SASET.  The  estimates  range  from  a  low  of  32 1 .3  MM  to  a  high  of  775. 15  MM.  The 
mean  value  is  496.83,  and  standard  deviation  is  135.46.  The  mean  value  will  be  used 
in  initial  economic  evaluation  and  the  standard  deviation  is  needed  for  risk  analysis.  A 
summary  of  the  estimates  of  effort  is  found  in  Figure  2. 
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Figure  2  Estimates  of  Effort  for  RESFMS  Redesign 
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h.  Software  Maintenance  Estimates 

Both  REVIC  and  SASET  allow  computation  of  the  level  of  effort  that 
will  be  required  to  maintain  the  system  during  its  extended  life-cycle.  These  values  are 
generally  expressed  in  terms  of  a  full-time  software  person  month  (FSP).  One  FSP  is 
the  equivalent  of  one  software  maintenance  person  working  one  full  month,  and  3.5  FSP 
would  be  the  equivalent  of  three  and  a  half  full-time  software  personnel  working  for  one 
month  on  the  project.  The  FSP  figures  can  be  multiplied  by  12  to  obtain  an  annual 
maintenance  figure  in  man-months  (MM)  and  this  figure  can  be  multiplied  by  the  cost 
per  MM  to  obtain  annual  maintenance  cost. 

Since  the  50%  Reuse  estimate  results  in  a  system  exactly  as  large  and 
complex  as  the  Low  LOC  estimate,  the  maintenance  required  on  such  systems  would  be 
the  same.  Also,  when  using  SASET,  though  dividing  LOC  estimates  for  modules  into 
support  and  applications  portions  resulted  in  different  estimates  of  development  effort 
than  lumping  them  together,  it  made  no  difference  in  the  resulting  maintenance  effort 
estimation.  Therefore,  four  sets  of  values  were  derived  for  FSP  requirements  for 
maintenance:  two  from  REVIC  and  two  from  SASET.  The  mean  and  standard  deviation 
of  the  four  estimates  was  calculated  for  each  year  of  maintenance.  The  mean  of  the 
maintenance  FSP  values  was  used  in  the  initial  economic  analysis  and  the  standard 
deviation  used  later  in  risk  analysis  computations.  Values  derived  for  maintenance  are 
displayed  in  Table  X. 
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Table  X  FSP  ESTIMATES  FOR  SOFTWARE  MAINTENANCE 


YEAR 

REVIC 

Low 

REVIC 

High 

SASET 

Low 

SASET 

High 

Mean 

Std  Dev 

1 

6.9 

9.8 

2.36 

3.74 

5.700 

3.32902 

2 

6.0 

8.5 

1.89 

2.99 

4.845 

2.99255 

3 

5.3 

7.5 

1.53 

2.43 

4.190 

2.73016 

4 

4.6 

6.6 

1.30 

2.06 

3.640 

2.42592 

5 

4.6 

6.6 

1.06 

1.68 

3.485 

2.58747 

6 

4.6 

6.6 

1.00 

1.59 

3.447 

2.62723 

7 

4.6 

6.6 

0.88 

1.40 

3.370 

2.70966 

8 

4.6 

6.6 

0.77 

1.21 

3.295 

2.78980 

i.  Hardware  Investment  and  Technical  Support 

The  target  computer  configuration  for  the  RESFMS  redesign  is  an 
AT&T  3B2/GR.  Earlier  analysis  determined  that  RESFMS  could  run  on  a  single 
3B2/GR  if  it  were  configured  with  at  least  14  GB  of  mass  storage,  but  that  the  number 
of  possible  simultaneous  uses  and  limitations  on  network  connections  per  computer  would 
indicate  that  two  3B2/GRs  would  be  optimum.  Also,  the  operation  of  two  computers  for 
the  same  application  would  allow  for  system  redundancy  and  backup  in  case  the  primary 
computer  fails. 

Vendor  representatives  for  the  AT&T  contract  provided  cost  figures  for 
the  computers  and  vendor  support  that  would  be  required  (NCR  letter,  22  June  1992). 
The  configuration  recommended  includes  the  following  items  for  each  computer 
purchased: 
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•  3B2/GR  computer  with  RISC  processor  running  at  25  MIPS 

•  64  Megabytes  of  RAM 

•  15  Gigabytes  of  hard  disk  storage 

•  Multi-Network  Processor  board  to  allow  connection  of  256  simultaneous  users 

•  Cables  and  connectors 

•  Storage  cabinets  for  all  equipment 

•  POSIX  operating  system  software  (a  form  of  UNIX) 

•  TCP/IP  network  software  for  UNIX  systems 

Total  cost  for  this  configuration  is  $129,892.47.  Two  computers  would 
thus  cost  $259,784.94.  These  computers  would  require  vendor  supplied  maintenance. 
Vendor  maintenance  for  the  hardware  would  cost  $3,344.09  per  year.  Maintenance 
support  for  the  software  (operating  system  and  network  software)  would  cost  $562.56  per 
year.  AT&T  will  supply  this  maintenance  support  for  up  to  five  years.  After  that  time 
support  may  be  purchased  from  other  vendors  at  a  cost  near,  or  below,  that  quoted  by 
AT&T  (Albro,  13  July  1992). 

COMNAVRESFOR  is  already  operating  a  network  of  3B2/GR 
computers  used  for  other  applications.  Sufficient  space,  air  conditioning  capacity,  and 
electrical  power  are  available  to  easily  add  four  to  six  new  computers  to  this  network. 
Addition  of  two  new  computers  for  RESFMS  would  not  require  any  additional 
construction,  nor  would  they  use  any  significant  amounts  of  additional  utilities  than  those 
already  in  operation.  Computer  room  operations  support  would  require  the  addition  of 
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the  equivalent  of  one  and  a  half  technical  support  personnel  to  meet  the  needs  of  the 
additional  workload.  Cost  of  technical  support  personnel  would  be  $68,000  per  person, 
including  overhead  and  fringe  benefits  (Albro,  13  July  1992).  Cost  of  technical  support 
for  the  new  computers  would  thus  be  $102,000  per  year. 

j.  Other  Operational  Costs  of  Redesign  Alternative 

NCTS  New  Orleans  currently  provides  the  service  of  a  help  desk  for 
RESFMS  support.  If  the  system  were  redesigned,  this  help  desk  function  would  have 
to  be  provided  by  COMNAVRESFOR.  The  help  desk  function  would  be  provided  by 
SEMA  contract  personnel  and  would,  therefore,  cost  the  Naval  Reserve  $4,852.40  per 
month  per  person.  Four  individuals  would  be  required  for  the  help  desk  function. 
Sufficient  telephone  and  office  space  already  exist  at  COMNAVRESFOR  facilities  to 
accommodate  this  function.  Therefore,  total  additional  cost  would  be  $232,915.20  per 
year. 

Management  and  support  personnel  attached  to  COMNAVRESFOR  and 
now  assigned  to  RESFMS  would  continue  this  function  with  the  redesigned  system.  No 
additional  tasking  would  be  required,  but  neither  would  their  workload  be  reduced. 
Supplies  and  travel  requirements  of  the  current  system  should  also  remain  fairly  constant 
if  a  redesigned  system  begins  operation.  Since  SEMA  contractors  are  physically  located 
in  COMNAVRESFOR  spaces,  and  system  requirements  and  design  are  already 
determined,  no  additional  travel  should  be  required  by  development  personnel. 
Therefore,  costs  for  COMNAVRESFOR  civilian  personnel,  supplies,  and  travel  are  the 
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same  as  those  found  in  the  current  system  and  would  begin  to  be  attributed  to  the 
redesigned  system  upon  initiation  of  system  operation. 

Software  tools  that  will  be  used  by  programmers  and  analysts  working 
on  the  redesign  have  either  already  been  purchased  or  are  available  from  other 
government  agencies  at  no  cost  to  the  Naval  Reserve.  COMNAVRESFOR  is  providing 
all  software  tools  for  SEMA  contract  programmers.  Much  of  the  initial  programming 
will  be  done  on  personal  computers  and  then  the  code  will  be  recompiled  for  the 
3B2/GR.  This  will  allow  much  greater  programmer  productivity,  since  the  3B2/GR  will 
only  have  to  be  shared  for  testing  of  finished  program  modules.  COMNAVRESFOR 
already  owns  sufficient  numbers  of  personal  computers  to  support  a  full  development 
effort  on  RESFMS  redesign  and  will  provide  these  computers  to  SEMA  for  their  use. 
Therefore,  no  additional  costs  will  be  incurred  for  either  software  tools  or  development 
computer  assets. 

B.  BENEFITS  CONSIDERED  FOR  RESFMS 

Even  though  benefits  for  RESFMS  will  not  be  formally  or  financially  analyzed,  it 
is  important  to  verify  that  the  benefits  of  the  redesigned  system  are  at  least  equivalent 
with  those  of  the  existing  system.  Otherwise,  any  cost  savings  that  may  accrue  from  a 
redesign  may  be  mitigated  by  reduced  functionality  or  reduced  benefits. 

A  briefing  document  prepared  for  COMNAVRESFOR  (CNRF  RESFMS  Briefing 
Paper,  August  1991)  has  identified  the  following  benefits  derived  from  the  current 
RESFMS  configuration: 


100 


•  Significantly  Improved  Management  of  RPN  Funds 

•  Fund  Utilization  Improved  from  98%  of  330M  in  FY82  to  99.97%  of  683  M  in 
FY87 

•  Allowed  Reuse  of  $5M  in  Travel  Funds  Recouped  from  Tickets  in  FY88/89 

•  Improved  Tracking  of  Active  Duty  Orders 

•  Reduced  Time  to  Generate  Orders 

•  Fully  Certified  Accounting  System  January  1987 

Since  the  redesigned  system  will  have  the  same  functional  requirements  as  the 
current  system,  and  will  be  programmed  by  personnel  currently  maintaining  RESFMS, 
there  is  ample  evidence  to  assume  that  benefits  of  the  current  system  will  be  present  in 
the  redesigned  system. 

The  primary  disadvantages  of  the  current  system  are: 

•  High  Cost  of  Operation 

•  Poor  Service  Provided  by  NCTS 

•  175  Reserve  Sites  of  354  Total  Not  Connected  to  RESFMS 

This  analysis  will  determine  if  costs  of  operation  are  less  for  the  redesigned  system. 
Service  will  no  longer  be  provided  by  NCTS,  but  taken  in  house  to  COMNAVRESFOR. 
With  more  direct  attention  and  close  control,  which  are  not  available  when  dealing  with 
a  separate  agency,  service  and  support  should  be  better  in  the  redesigned  system. 
Finally,  the  redesign  includes  a  new  user  interface  that  will  be  designed  to  allow 
connection  to  RESFMS  by  all  354  Reserve  sites.  Since  benefits  of  the  redesigned  system 
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should  be  equal  or  greater  than  the  current  system,  a  straight  comparison  of  costs  should 
be  sufficient  for  the  redesign  decision. 

C.  PROJECT  LIFE 

As  stated  earlier,  economic  life  is  that  period  over  which  savings  and  benefits  are 
expected  to  accrue;  and  the  economic  life  chosen  for  analysis  of  alternatives  is  usually 
the  shortest  of  technological,  mission,  or  physical  life  (Haga  and  Lang,  1990). 
Sometimes  it  is  necessary  to  make  investments  several  years  prior  to  the  beginning  of  the 
economic  life,  which  is  when  the  project  begins  producing  savings  or  benefits.  The 
period  between  initial  investment  and  beginning  of  economic  life  is  called  "lead  time" 
(Haga  and  Lang,  1990).  Project  life  is  the  combination  of  lead  time  and  economic  life. 

In  the  case  of  RESFMS,  if  new  AT&T  3B2/GR  computers  are  purchased  they  will 
have  a  physical  life  of  eight  to  ten  years.  If  the  Unisys  1100/92  is  retained,  it  will 
probably  have  a  similar  physical  life  because  the  system  was  only  recently  upgraded  from 
an  1 100/90  to  the  1 100/92.  Mission  life  is  indefinite,  since  the  function  being  performed 
is  essential  to  Naval  Reserve  operations  and  will  continue  to  be  a  need  for  the  foreseeable 
future.  Technological  life  is  more  difficult  to  determine.  Even  though  available 
technology  may  quickly  and  dramatically  improve,  both  the  3B2/GR  and  the  Unisys 
1100/92  are  adequate  for  the  task  at  present.  The  question  becomes  one  of 
maintainability  and  cost-effectiveness  of  technological  replacement.  NCTS  claims  that 
they  can  provide  effective  maintenance  indefinitely  for  the  Unisys  1 100/92  or  any  chosen 
NCTS  operated  follow-on  computer.  AT&T  will  provide  full  maintenance  support  for 


102 


the  3B2/GR  for  five  years.  A  number  of  other  contractors  are  available  to  provide 
maintenance  after  five  years.  Therefore,  a  ten  year  hardware  system  life  is  feasible. 

RESFMS  has  already  been  operational  for  nine  years.  Investment  in  software 
development  began  two  years  before  the  first  module  was  operational  making  RESFMS 
project  life,  to  date,  eleven  years.  The  Director  of  Defense  Information  has  stated  that 
one  goal  of  CIM  is  to  achieve  20  year  and  greater  system  life  for  DoD  information 
systems.  Therefore,  an  additional  ten  years  of  project  life  for  RESFMS  in  its  present 
configuration  is  also  feasible. 

Estimates  of  schedule  for  software  development  of  the  redesign  alternative  vary 
from  20  to  29.8  months.  SASET  has  been  more  thoroughly  tested  and  calibrated  for 
development  projects  in  the  Department  of  Defense  and  is,  therefore,  probably  more 
reliable  (Gross,  5  May  1992).  The  schedule  estimates  from  SASET  ranged  from  20  to 
25  months.  It  would  be  reasonable  to  assume  that,  if  ample  resources  are  applied  and 
an  optimum  schedule  pursued,  software  development  for  the  redesign  would  probably 
require  approximately  two  years.  If  however,  sufficient  funding  is  not  initially  made 
available  or  the  high  development  man-month  estimates  prove  correct,  a  three  year  lead 
time  may  be  required.  This  means  that,  for  this  economic  analysis,  there  will  be  a  lead 
time  of  two  or  three  years  before  a  new  system  will  begin  generating  savings  and 
benefits.  During  this  time  the  current  configuration  will  continue  to  operate. 

Considering  all  these  factors,  I  have  chosen  a  project  life  of  ten  years  due  to  the 
physical  life  limit  of  the  3B2/GR  computers.  Economic  life  starts  when  the  system 
begins  to  generate  savings  or  benefits.  Thus,  the  economic  life  will  be  either  seven  or 


103 


eight  years  depending  on  whether  a  three  or  two  year  lead  time  is  used.  Both  scenarios 
will  be  considered  and  results  for  each  calculated  in  the  analysis  of  alternatives  for 
RESFMS. 
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VII.  ANALYSIS 


A.  PROJECT  LIFE  CYCLE 

The  project  life  cycle  will  begin  with  the  purchase  of  at  least  one  AT&T  3B2/GR 
computer.  In  order  to  provide  adequate  computer  resources  for  testing 
COMNAVRESFOR  desires  to  have  two  3B2/GR  computers  dedicated  to  the  development 
effort  (Furrey,  14  May  1992).  Since  it  will  be  necessary  to  test  software  on  the  target 
computer  platform  from  the  earliest  stages  of  development,  the  purchase  and  installation 
of  two  3B2/GR  computers  should  take  place  at  the  beginning  of  YEAR  ONE  of  project 
life.  Development  should  take  two  or  three  years,  during  which  software  development 
costs  and  3B2/GR  technical  support  will  be  needed.  The  current  configuration  of 
RESFMS  will  continue  to  operate  during  this  lead  time.  However,  costs  of  operating  the 
current  configuration  will  not  be  considered  during  this  lead  time  because  these  costs  will 
be  incurred  regardless  of  which  alternative  is  selected.  The  first  year  in  which  a  system 
begins  generating  benefits  is  the  beginning  of  economic  life.  Economic  life  will  begin 
in  YEAR  THREE  of  project  life,  if  a  two  year  lead  time  is  used,  or  YEAR  FOUR,  if 
a  three  year  lead  time  is  used.  Therefore,  the  economic  life  for  comparison  will  be  from 
YEAR  THREE  to  YEAR  TEN  of  project  life  in  the  case  of  two  year  lead  time,  and  from 
YEAR  FOUR  to  YEAR  TEN  of  project  life  in  the  case  of  three  year  lead  time.  Salvage 
value  will  not  be  considered  in  this  analysis  as  it  is  anticipated  that  technological 
advances  in  the  next  ten  years  will  render  salvage  value  of  the  computer  equipment  zero. 
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B.  DISCOUNTED  COSTS 


Department  of  Defense  directives  stipulate  that  costs  will  first  be  calculated  in 
constant  year  dollars  and  that  a  ten  percent  discount  factor  will  then  be  applied  to 
calculate  the  present  value  of  future  costs. 

One  way  to  compute  the  present  value  of  costs  is  to  use  a  formula  and  compute  the 
exact  discount  factor  for  the  time  the  cost  is  incurred.  For  instance,  if  costs  occurred 
monthly,  the  formula  could  be  used  to  calculate  a  different  discount  factor  for  each 
month.  This  method  is  very  tedious  and  time  consuming.  If  costs  are  evenly  distributed 
throughout  the  year,  a  similar  result  can  be  obtained  by  simply  averaging  the  discount 
factors  for  the  beginning  and  the  end  of  year  to  determine  a  mid-year  factor  and  applying 
this  factor  to  the  total  costs  for  the  year. 

A  table  of  ten  percent  discount  factors  may  be  found  in  Appendix  B.  These  are 
mid-year  factors  and  their  use  assumes  an  investment  or  cost  is  evenly  distributed 
throughout  the  year.  For  costs  or  investments  which  occur  at  a  particular  point  in  time, 
not  distributed  throughout  a  year,  a  different  set  of  discount  factors  should  be  used. 

In  the  case  of  RESFMS  only  the  cost  of  initial  purchase  of  the  3B2/GR  computers 
occurs  at  a  single  point  in  time,  at  the  beginning  of  YEAR  ONE.  All  other  costs  can  be 
assumed  to  be  evenly  distributed  throughout  the  year  in  which  they  occur.  Therefore  the 
discount  factors  found  in  Appendix  B  may  be  applied  to  all  costs  in  this  analysis  except 
the  cost  of  initial  investment  for  3B2/GR  computers.  Since  the  purchase  of  3B2/GRs 
comes  at  the  beginning  of  YEAR  ONE  the  present  value  of  that  cost  will  be  exactly  the 
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same  as  the  cost  itself.  In  other  words,  a  discount  factor  of  1.0  is  appropriate  for  this 
initial  investment. 

One  way  to  represent  the  fact  that  initial  investments  occur  prior  to  the  evenly 
distributed  costs  incurred  in  YEAR  ONE  is  to  place  them  in  a  column  labeled  YEAR 
ZERO.  The  discount  factor  applied  then  in  YEAR  ZERO  is  1.0  and  the  factor  applied 
in  YEAR  ONE  and  subsequent  years  is  the  mid-year  factor  from  Appendix  B.  This  is 
the  method  suggested  by  Haga  and  Lang  (1991)  and  tne  method  which  will  be  used  in 
this  analysis. 

C.  PRESENT  VALUE 

Present  Value  analysis  was  done  comparing  the  Status  Quo  and  Redesign 
alternatives  considering  both  two  year  and  three  year  lead  times.  Results  of  Present 
Value  analyses  for  the  two  year  lead  time  assumption  are  summarized  in  Table  XI. 
Results  of  Present  Value  analyses  for  the  three  year  lead  time  assumption  are  summarized 
in  Table  XII. 

Table  XI  PRESENT  VALUE  -  TWO  YEAR 
LEAD  TIME 

Net  Present  Value 

Status  Quo  $21,9969,150 

Redesign  7,653,445 

Savings  $14,315,705 
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Table  XU  PRESENT  VALUE  -  THREE  YEAR 
LEAD  TIME 


Status  Quo 

Redesign 

Savings 


Tables  XIII  through  XX  show  the  detailed  analysis  of  the  net  present  value  of  both 
the  Status  Quo  and  Redesign  alternatives.  Each  Table  showing  a  ten  year  cash  flow  has 
been  split  into  two  tables  for  ease  in  reading.  In  this  analysis,  the  expected  value  for  the 
following  variables  was  used  and  was  computed  using  the  type  of  distribution  indicated: 

•  Status  Quo  Software  Maintenance  -  Triangular  Distribution 

•  NIF  Charges  to  NCTS  -  Triangular  Distribution 

•  Man-Months  of  Development  Effort  -  Normal  Distribution 

•  Redesign  Annual  Software  Maintenance  -  Normal  Distribution 

Values  were  derived  as  described  in  Chapter  VI. 
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Table  XIII  PRESENT  VALUE  OF  STATUS  QUO  -  TWO  YEAR  LEAD  TIME  -  YEAR  ZERO  TO  YEAR  FIVE 
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Table  XIV  PRESENT  VALUE  OF  STATUS  QUO  -  TWO  YEAR  LEAD  TIME  -  YEAR  SIX  TO  YEAR  TEN 


no 


j 


Table  XV  PRESENT  VALUE  OF  REDESIGN  -  TWO  YEAR  LEAD  TIME  -  YEAR  ZERO  TO  YEAR  FIVE 
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Table  XVI  PRESENT  VALUE  OF  REDESIGN  -  TWO  YEAR  LEAD  TIME  -  YEAR  SIX  TO  YEAR  TEN 
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PRESENT  VALUE  Of  REDESIGN  | _  _  S7.6S3.445 


Table  XVII  PRESENT  VALUE  OF  STATUS  QUO  -  THREE  YEAR  LEAD  TIME  -  YEAR  ZERO  TO  YEAR  FIVE 
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Table  XVIII  PRESENT  VALUE  OF  STATUS  QUO  -  THREE  YEAR  LEAD  TIME  -  YEAR  SIX  TO  YEAR  TEN 


Table  XIX  PRESENT  VALUE  OF  REDESIGN  -  THREE  YEAR  LEAD  TIME  -  YEAR  ZERO  TO  YEAR  FIVE 
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Table  XX  PRESENT  VALUE  OF  REDESIGN  -  THREE  YEAR  LEAD  TIME  -  YEAR  SIX  TO  YEAR  TEN 


PRESENT  VALUE  OF  REDESIGN  I  _ _ _ _ _ >6,804,361 


D.  SAVINGS/INVESTMENT  RATIO 

The  SIR  shows  the  “relationship  between  future  cost  savings  and  the  investment 
necessary  to  obtain  those  savings "  (Haga  and  Lang,  1991).  Savings  only  accrue  during 
the  economic  life  of  the  project.  Investment  is  normally  made  during  the  lead  time 
period.  In  the  case  of  RESFMS,  savings  begin  to  accrue  in  YEAR  THREE  (two  year 
lead  time)  or  YEAR  FOUR  (three  year  lead  time)  and  continue  to  YEAR  TEN.  In  each 
of  these  years,  the  current  year  value  of  the  Redesign  Alternative  is  subtracted  from  the 
current  year  value  of  Status  Quo  costs  to  determine  annual  savings.  The  appropriate 
discount  factor  from  APPENDIX  B  is  then  applied  to  obtain  the  present  values  of  annual 
savings.  The  present  value  of  savings  for  each  year  in  the  economic  life  are  summed  to 
form  the  present  value  of  total  savings,  PVS. 

In  a  similar  manner,  the  current  year  costs  of  investments  are  totaled.  Then  the 
appropriate  discount  factor  is  applied  to  determine  present  value  of  each  year  of 
investment.  The  sum  of  all  investment  years  produces  the  present  value  of  total 
investment,  PV,.  To  determine  the  SIR,  the  following  formula  is  used: 


Tables  XXI  and  XXII  detail  the  computations  for  PVS  and  PV,;  first,  in  the  case 
of  two  year  lead  time,  and  then,  for  a  three  year  lead  time. 
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Table  XXI  PRESENT  VALUE  OF  SAVINGS  AND  INVESTMENTS  -  TWO  YEAR  LEAD  TIME 


CUMUL AT  I VE  PV  OP  INVESTMENT  I _ $259,785 
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PV  OF  AHMUAL  INVESTMENT  I _ >259.785  I _ >867.675  I _ >788.548  | _ >716.696 


The  resulting  values  for  PVS  and  PV,  are  then  used  to  compute  the  SIR.  Thus  the 


SIR  computation  for  the  two  year  lead  time  situation  is  as  follows: 


sir2 


PVs  =  $16,963,397  =  g  4Q7 
PVj.  $2,647,691 


The  SIR  computation  for  the  three  year  lead  time  situation  is  as  follows: 


sir2 


PVs 
PV x 


$14,055,236  =  5  339 
$2,632,704 


E.  DISCOUNTED  PAYBACK  ANALYSIS 

In  order  to  compute  discounted  payback  (PB),  the  present  value  of  investment  and 
an  annual  savings  figure  are  required.  In  many  cases  the  annual  savings  figure  is 
constant  throughout  the  economic  life.  However,  in  the  case  of  RESFMS,  the  annual 
savings  figure  increases,  in  constant  year  dollars,  each  successive  year  of  economic  life. 
Therefore,  for  this  analysis  the  savings  from  the  first  year  of  economic  life  will  be  used, 
since  the  first  year  of  savings  will  be  the  lowest  annual  amount  and  thus  yield  the  worst 
case  value  of  the  payback  factor. 

To  determine  discounted  payback,  the  present  value  of  investment,  PV,,  is  divided 
by  the  annual  savings  in  constant  year  dollars.  To  this  result  is  added  the  cumulative 
discount  factor  from  APPENDIX  B,  Table  B  for  the  number  of  years  of  lead  time. 
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When  the  final  value  obtained  is  compared  with  APPENDIX  B,  Table  B  cumulative 
discount  factors,  the  year  in  which  payback  occurs  can  be  determined. 

For  RESFMS,  payback  was  calculated  for  both  two  and  three  year  lead  time.  In 
the  case  of  two  year  lead  time,  the  cumulative  factor  is  computed  as  follows: 

_ Pl± _  =  $2,647,691  =  0  740 

Annual  Savings  for  YELAR  3  $3,577,333 

This  value  is  then  adjusted  for  lead  time  by  adding  the  cumulative  factor  from 
APPENDIX  B  for  two  years. 

0.740  +  1.821  =  2.561 


Comparison  with  APPENDIX  B,  Table  B  shows  that  2.561  falls  between  the  YEAR 
TWO  cumulative  value  of  1.821  and  the  YEAR  THREE  cumulative  value  of  2.609. 
This  means  that  payback  occurs  within  YEAR  THREE,  or  before  the  end  of  the  first 
year  of  economic  life. 

In  the  case  of  three  year  lead  time,  the  cumulative  factor  is  computed: 

_ _  =  $2,632,704  =  0  736 

Annual  Savings  for  YEAR  4  $3,577,333 

This  value  is  then  adjust  for  lead  time  by  adding  the  cumulative  factor  from  APPENDIX 
B,  Table  B  for  three  years. 
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0.736  +  2.609  =  3.345 


Comparison  with  APPENDIX  B,  Table  B  indicates  that  3.345  falls  between  the  YEAR 
FOUR  cumulative  value  of  3.326  and  the  YEAR  FIVE  cumulative  value  of  3.977.  Thus 
payback  occurs  shortly  after  the  beginning  of  YEAR  FOUR,  or  in  the  second  year  of 
economic  life. 

F.  RISK  ANALYSIS 

In  order  to  assess  the  amount  of  risk  involved  in  this  economic  analysis,  and  to 
determine  most  probable  outcomes,  a  financial  simulation  was  conducted  using  ©RISK. 
©RISK  is  a  risk  analysis  and  modeling  tool,  produced  by  Palisade  Corporation, 
Newfield,  N.Y.10  It  functions  as  an  add-in  program  to  Lotus  1-2-3”. 

Using  a  Monte  Carlo  type  simulation,  an  analysis  tool  such  as  ©RISK  can  allow 
decision  makers  to  determine  relative  interaction  between  variables,  and  possible  ranges 
of  values  that  may  be  expected  from  project  execution.  Probability  of  positive  outcomes 
may  also  be  determined. 

In  the  case  of  RESFMS,  four  types  of  values  were  varied  for  purposes  of  the 
simulation.  In  the  Status  Quo  alternative,  Software  Maintenance  and  NIF  Charges  to 
NCTS  were  both  varied  using  a  Triangular  distribution  as  shown  in  Table  XXIII. 


10The  particular  version  of  ©RISK  used  for  this  simulation  was  the  Student  Edition 
of  ©RISK,  Version  1.55,  distributed  by  Addison-Wesley  Publishing  Company,  Inc. 

1 ’Lotus  1-2-3  is  a  registered  trademark  of  Lotus  Development  Corporation. 
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Table  XXIII  STATUS  QUO  VARIABLE  COSTS 


Variable  Cost:  High  Low  Most  Probable 

Software  Maintenance  $1,608,000  $524,124  $1,050,000 

NIF  Charges  to  NCTS  $3,514,000  $2,841,000  $3,040,000 


In  the  Redesign  alternative,  development  effort  in  Man-Months,  and  annual  maintenance 
FSP  were  varied  using  a  Normal  distribution.  An  examination  of  software  maintenance 
FSP  projections  shows  that,  even  though  the  estimates  vary  widely  between  REVIC  and 
SASET  outputs,  the  values  are  always  higher  in  the  early  years  and  taper  off  as  project 
life  progresses.  To  insure  this  relationship  was  consistently  portrayed  in  the  simulation, 
the  first  year  FSP  value  was  made  an  Independent  variable  and  all  subsequent  years  of 
maintenance  effort  were  made  Dependent  variables.  Although  all  years  of  maintenance 
effort  varied  individually,  the  trend  of  the  Independent  variable  was  used  to  determine 
the  trend  of  the  Dependent  variables.  For  instance,  if  the  value  for  the  first  year  of 
maintenance,  in  any  single  iteration  of  the  simulation,  was  below  the  mean,  then  values 
for  subsequent  years  of  maintenance  would  be  chosen  from  below  the  mean.  This 
assured  the  proper  cost  trends  would  be  simulated.  Table  XXIV  below  lists  variables  in 
the  redesign  alternative  and  values  used  for  simulation. 

Monte  Carlo  type  simulation  was  done  using  4,000  iterations  for  the  two  year  lead 
time  scenario.  Due  to  computer  time  constraints,  a  simulation  using  1 ,000  iterations  was 
done  for  the  three  year  lead  time  scenario.  Results  of  the  simulation  were  calculated  for 
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Table  XXIV  REDESIGN  ALTERNATIVE  VARIABLES 


Variable  Value 

Man-Months  of  Development 

Year  1  FSP 

Year  2  FSP 

Year  3  FSP 

Year  4  FSP 

Year  5  FSP 

Year  6  FSP 

Year  7  FSP 

Year  8  FSP 


Mean 

Standard  Deviation 

496.83 

135.46 

5.7000 

3.32902 

4.8450 

2.99255 

4.1900 

2.73016 

3.6400 

2.42592 

3.4850 

2.58747 

3.4475 

2.62723 

3.3700 

2.70966 

3.2950 

2.78980 

Man-Months  of  Development  Effort,  Present  Value  (PV)  of  Savings,  Savings/Investment 
Ratio  (SIR),  and  Discounted  Payback  Factor  (PB). 

Results  of  the  simulations  in  the  form  of  histograms  and  associated  statistics  are 
displayed  on  the  following  pages.  The  distribution  of  values  and  probabilities  for  Man- 
Months  of  Development  is  identical  for  both  two  year  and  three  year  lead  time.  The 
resulting  cost  is  simply  distributed  evenly  over  the  number  of  years  of  lead  time. 
Therefore,  the  distribution  for  Man-Months  of  Effort  is  only  displayed  once,  in  Figure 
3  and  Table  XXV.  The  results  for  the  the  two  year  lead  time  scenario  follow.  PV  of 
Savings  is  displayed  in  Figure  4  and  Table  XXVI,  SIR  in  Figure  5  and  Table  XXVII, 
and  PB  in  Figure  6  and  Table  XXVIII.  Then  the  results  of  the  three  year  lead  time  are 
displayed.  PV  of  Savings  is  displayed  in  Figure  7  and  Table  XXIX,  SIR  in  Figure  8  and 
Table  XXX,  and  PB  in  Figure  9  and  Table  XXXI. 
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Figure  3  Man-Months  of  Effort  for  RESFMS  Software  Redesign 


Table  XXV  RISK  ANALYSIS  STATISTICS  FOR  MAN- 
MONTHS  OF  EFFORT 


Expected/Mean  Result 

=  496.8263 

Maximum  Result 

=  968.7614 

Minimum  Result 

=  1.12505 

Range  of  Possible  Results 

=  967.6363 

Standard  Deviation 

=  135.4097 

Iterations 

=  4000 
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Table  XXVI  RISK  ANALYSIS  STATISTICS  FOR  PRESENT  VALUE 
OF  SAVINGS  -  TWO  YEAR  LEAD  TIME 


Expected/Mean  Result 

=  14,315,660 

Maximum  Result 

=  19,968,080 

Minimum  Result 

=  8,915,915 

Range  of  Possible  Results 

=  11,052,170 

Probability  of  Positive  Result 

=  100% 

Probability  of  Negative  Result 

=  0% 

Standard  Deviation 

=  1,637,532 

Iterations 

=  4,000 
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Figure  5  Savings/ Investment  Ratio  -  Two  Year  Lead  Time 


Table  XXVII  RISK  ANALYSIS  STATISTICS  FOR 
SAVINGS/INVESTMENT  RATIO  --  TWO  YEAR  LEAD  TIME 


Expected/Mean  Result 

=  6.797859 

Maximum  Result 

=  39.27233 

Minimum  Result 

=  3.214479 

Range  of  Possible  Results 

=  36.05785 

Probability  of  Positive  Result 

=  100% 

Probability  of  Negative  Result 

=  0% 

Standard  Deviation 

=  2.008756 

Iterations 

=  4,000 

<  CD 


Figure  6  Discounted  Payback  Factor  -  Two  Year  Lead  Time 

Table  XXVIII  RISK  ANALYSIS  STATISTICS  FOR  DISCOUNTED 
PAYBACK  FACTOR  -  TWO  YEAR  LEAD  TIME 


Expected/Mean  Result 

=  2.567812 

Maximum  Result 

=  3.318428 

Minimum  Result 

=  1.941221 

Range  of  Possible  Results 

=  1.377207 

Probability  of  Positive  Result 

=  100% 

Probability  of  Negative  Result 

=  0% 

Standard  Deviation 

=  0.1837197 

Iterations 

=  4,000 
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Figure  7  Present  Value  of  Savings  -  Three  Year  Lead  Time 


Table  XXIX  RISK  ANALYSIS  STATISTICS  FOR  PRESENT  VALUE 
OF  SAVINGS  -  THREE  YEAR  LEAD  TIME 


Expected/Mean  Result 

=  11,422,680 

Maximum  Result 

=  15,543,950 

Minimum  Result 

=  6,527,336 

Range  of  Possible  Results 

=  9,016,610 

Probability  of  Positive  Result 

=  100% 

Probability  of  Negative  Result 

=  0% 

Standard  Deviation 

=  1,407,193 

Iterations 

=  1,000 
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Figure  8  Savings/ Investment  Ratio  -  Three  Year  Lead  Time 


Table  XXX  RISK  ANALYSIS  STATISTICS  FOR 
SAVINGS/INVESTMENT  RATIO  -  THREE  YEAR  LEAD  TIME 


Expected/Mean  Result 

=  5.63609 

Maximum  Result 

=  19.34061 

Minimum  Result 

=  2.862788 

Range  of  Possible  Results 

=  16.47782 

Probability  of  Positive  Result 

=  100% 

Probability  of  Negative  Result 

=  0% 

Standard  Deviation 

=  1.567336 

Iterations 

=  1,000 

<  CO 


Figure  9  Discounted  Payback  Factor  -  Three  Year  Lead  Time 

Table  XXXI  RISK  ANALYSIS  STATISTICS  FOR  DISCOUNTED 
PAYBACK  FACTOR  -  THREE  YEAR  LEAD  TIME 


Expected/ Mean  Result 

=  3.352199 

Maximum  Result 

=  4.01652 

Minimum  Result 

=  2.808435 

Range  of  Possible  Results 

=  1.208085 

Probability  of  Positive  Result 

=  100% 

Probability  of  Negative  Result 

=  0% 

Standard  Deviation 

=  0.1789016 

Iterations 

=  1,000 
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G.  FINDINGS 


1.  Comparison  of  results 

In  the  economic  analysis  using  expected  values  of  variables,  all  three 
economic  analysis  tools  used  produced  results  favoring  the  redesign  alternative.  Using 
Net  Present  Value  analysis,  present  value  of  the  savings  were  positive.  The  three  year 
lead  time  yielded  the  lower  value,  which  was  still  in  excess  of  $11  million  over  the  life 
of  the  project. 

A  SIR  greater  than  1.0  should  be  obtained  before  a  project  should  be 
considered  economically  sound  (Haga  and  Lang,  1991).  In  the  case  of  RESFMS, 
expected  value  computations  indicate  a  SIR  of  5.339  for  the  three  year  lead  time 
alternative,  and  6.407  if  development  is  completed  in  two  years. 

Discounted  payback  analysis  shows  that  payback  is  expected  to  occur  within 
the  first  two  years  of  economic  life.  For  the  two  year  lead  time  scenario,  payback  was 
within  the  first  year  of  economic  life.  In  the  three  year  scenario,  payback  is  expected 
to  occur  soon  after  the  beginning  of  the  second  year  of  economic  life.  This  is  a  short 
payback  period  and  indicates  a  strong  economic  investment. 

2.  Risk  Analysis 

Examination  of  histograms  and  related  statistics  for  financial  simulation 
performed  for  the  RESFMS  project  supports  the  positive  results  of  expected  value 
computations  discussed  above. 
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In  simulation  of  both  two  and  three  year  lead  time  scenarios,  PV  of  savings 
remained  positive  for  all  iterations  of  the  simulation.  Probability  of  a  positive  result  was 
100%  for  both  scenarios.  The  minimum  result  obtained  by  simulation  was  $6,527,336, 
calculated  in  the  three  year  lead  time  scenario.  Examining  the  histograms,  we  find  the 
cumulative  probability  for  a  PV  of  savings  value  greater  than  $11  million  is  more  than 
90%  for  the  two  year  lead  time  and  about  60%  for  the  three  year  lead  time.  Therefore, 
regardless  of  lead  time  scenario  chosen,  PV  of  savings  can  be  expected  to  be  no  less  than 
$6.5  million  and  there  is  more  than  a  60%  percent  chance  that  it  will  be  greater  than  $11 
million. 

Worst  case  for  SIR  in  the  simulations  still  shows  a  ratio  of  2.86  in  the  three 
year  lead  time  scenario.  Probabilities  indicate  that  a  value  above  5.0  is  much  more 
likely.  Thus,  even  in  the  worst  case,  SIR  would  indicate  the  redesign  alternative  is  an 
economically  sound  project  to  pursue. 

Simulation  results  for  discounted  payback  analysis  show  that  payback  will 
probably  occur  within  two  years  of  project  implementation  regardless  of  whether  lead 
time  is  two  or  three  years.  The  worst  case  found  in  the  simulations  was  a  value  of  4.016 
in  the  three  year  lead  time  scenario.  Even  this  very  low  probability  result  would  have 
payback  occur  within  the  third  year  after  the  beginning  of  economic  life.  Since  economic 
life  is  expected  to  last  seven  years  in  the  three  year  lead  time  scenario,  savings  beyond 
the  value  of  initial  investment  would  still  accrue  for  at  least  three  years  more. 

Therefore,  risk  analysis  shows  no  probability  of  negative  values,  given  the 
range  of  values  and  assumptions  made  in  this  analysis.  I  have  endeavored  to  accurately 
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portray  ranges  of  possible  values,  to  include  all  possible  costs,  and  to  completely  quantify 
significant  economic  risk.  As  a  result,  I  believe  the  risk  analysis  simulations  performed 
are  a  reliable  indicator  of  potential  outcomes. 

3.  Patterns  Discerned 

All  results  obtained  were  positive.  There  were  no  negative  results.  Even  the 
worst  case  results  of  simulations  show  PV  of  savings,  SIR,  and  payback  periods  well 
within  acceptable  ranges  for  project  approval.  All  these  results  affirm  the  economic 
prudence  of  pursuing  the  redesign  of  RESFMS.  Baring  unforseen  technical  problems  that 
significantly  alter  development  effort  estimates,  RESFMS  redesign  should  be 
economically  successful. 
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VIH.  RECOMMENDATION 


A.  RESULTS  OF  ECONOMIC  ANALYSIS 

The  results  of  all  economic  analyses  indicate  a  favorable  economic  outcome  if  the 
RESFMS  redesign  is  implemented.  All  factors  considered  in  this  analysis  favor  the 
redesign  over  the  status  quo.  A  preliminary  technical  analysis  reveals  no  significant 
areas  of  concern,  involving  hardware,  resulting  from  the  move  of  RESFMS  from  a 
mainframe  environment  to  a  minicomputer.  The  primary  expense,  and  greatest  effort 
required,  will  come  from  rewriting  software  code.  In  spite  of  the  difficulty  encountered 
in  deriving  an  estimate  of  the  cost  of  software  development,  I  believe  that  a  reasonable 
range  of  potential  values  was  produced  and  adequate  risk  analysis  performed  to  ensure 
that  potential  costs  were  reliably  represented. 

Since  the  results  of  all  tools  used  and  of  all  simulations  performed  strongly  indicate 
the  economic  soundness  of  the  redesign  alternative,  I  recommend  that  the  redesign  of 
RESFMS  to  operate  in  the  configuration  described  in  this  analysis  be  approved  and 
initiated.  It  is  significant  to  note  that  greater  savings,  higher  SIR,  and  shorter  payback 
were  all  predicted  for  the  two  year  lead  time  than  for  the  three  year  lead  time  scenario. 
Actual  schedule  duration  for  the  redesign  development  effort  will  be  determined  by  the 
level  of  funding  made  available  for  the  project.  If  funding  is  low  and  spread  out  over 
several  years,  fewer  programmers  will  be  assigned  and  project  development  will  be 
delayed.  Although  this  analysis  indicates  that  positive  results  will  still  be  achieved,  those 
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results  will  be  less  than  can  be  obtained  if  wholehearted  approval  is  given  and  adequate 
resources  are  allocated  to  allow  a  short  development  time. 

B.  OTHER  CONSIDERATIONS 

Even  though  the  redesign  of  RESFMS  would  mean  significant  savings  to  the  Naval 
Reserve,  there  are  two  factors  which  may  prevent  the  development  effort  from  being 
approved.  First,  monies  to  operate  systems  and  monies  to  develop  systems  do  not 
necessarily  come  from  the  same  appropriation.  Also,  there  are  very  specific  approval 
procedures  required  for  development  efforts  that  do  not  apply  to  currently  operating 
systems.  In  particular,  the  nearly  $300,000  required  to  purchase  two  3B2/GR 
minicomputers  would  necessarily  come  from  the  Other  Procurement  Navy  (OPN) 
appropriation  and  would  require  special  approval  before  it  could  be  spent  (Practical 
Comptrollership,  1992).  Also,  if  the  cost  of  software  development  is  not  competitively 
bid  and  exceeds  $250,000,  or  is  competitively  bid  and  exceeds  $2.5  million,  an  Agency 
Procurement  Request  must  be  sent  to  GSA  for  approval  (GSA  Overview  Guide,  January 
1990).  These  additional  requirements  and  constraints  might  cause  approval  to  be 
delayed,  or  the  project  to  be  canceled  as  the  Naval  Reserve  competes  with  other  DoD 
agencies  for  Information  System  development  and  acquisition  funds.  COMNAVRESFOR 
personnel  must  effectively  communicate  the  significant  savings  that  a  redesigned 
RESFMS  can  produce  and  as  a  result  obtain  funding  and  approval  required  to  continue. 

The  second  consideration  is  that,  although  moving  RESFMS  to  minicomputers 
controlled  by  COMNAVRESFOR  may  produce  great  savings  for  the  Naval  Reserve,  it 
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may  not  necessarily  save  the  Department  of  the  Navy  anything  at  all.  In  fact,  depending 
on  what  NCTS  does  in  response,  it  may  actually  cost  DoN  more  than  the  status  quo. 
This  is  due  to  the  fact  that  NCTS  is  a  Navy  owned  and  operated  organization.  Thus, 
NCTS  funding  comes  from  DoN  budgets  just  as  COMNAVRESFOR  funding  does.  If 
NCTS  is  charging  COMNAVRESFOR  based  on  actual  costs  of  maintaining  their  facility, 
and  if  no  replacement  customers  are  found  when  COMNAVRESFOR  moves,  these 
overhead  costs  may  still  be  incurred  by  the  Navy  as  a  whole  but  no  longer  charged  to  the 
Naval  Reserve.  Of  course,  a  reasonable  person  would  conclude  that  if  NCTS  is  not 
providing  cost  effective  service  then  the  Navy  should  reconsider  its  support.  In  any  case, 
the  actions  of  NCTS  are  beyond  the  scope  of  this  analysis.  What  is  clear  from  this 
analysis  is  that  the  Naval  Reserve  can  save  significant  amounts  of  future  operating 
expense  if  RESFMS  is  redesigned.  If  all  Navy  activities  pursued  projects  that  resulted 
in  operating  expense  savings,  the  Navy  as  a  whole  would  have  to  benefit.  Theiefore,  the 
redesign  of  RESFMS  should  be  evaluated  on  its  own  merits  and  a  development  decision 
should  not  be  affected  by  short  term  effects  on  other  service  agencies. 

C.  CONCLUSION 

This  economic  analysis  indicates  that  a  clear  economic  advantage  to  the  Naval 
Reserve  will  result  from  a  redesign  of  RESFMS.  It  logically  follows  that  the  Department 
of  the  Navy  will  ultimately  benefit  from  reduced  cost  in  funding  the  Naval  Reserve.  To 
gain  maximum  savings,  adequate  resources  should  be  allocated  to  develop  the  new 
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system  in  minimum  time.  Therefore,  the  Naval  Reserve  should  pursue  a  redevelopment 
effort  and  DoN  fully  support  the  effort  to  allow  early  realization  of  projected  savings. 
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APPENDIX  A 


ADSR 

ADPE 

ADT 

AIS 

AT 

BCR 

BE 

bps 

CASE 

CERPS 

CINCLANTFLEET 

COMNAVRESFOR 

CNRF 

CNRF  Code  02 
CNRF  Code  06 

CNRF  Code  10 


ACRONYMNS  AND  ABBREVIATIONS 

Automated  Data  Service  Request 
Automated  Data  Processing  Equipment 
Active  Duty  Training 
Automated  Information  System 
Annual  Training 
Benefit/Cost  Ratio 
Break  Even 
bits  per  second 

Computer  Aided  Software  Engineering 

Centralized  Expenditure  Register  Processing  System 

Commander  in  Chief,  U.S.  Atlantic  Fleet 

Commander  Naval  Reserve  Force 

Commander  Naval  Reserve  Force 

Commander  Naval  Reserve  Force  Director  of  Finance 

Commander  Naval  Reserve  Force  Director  of  Manpower 

and  Personnel 

Commander  Naval  Reserve  Force  Director  of  Information 
Systems 
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CPU 

DBOF 

DDI 

DDN 

DoD 

DODD 

DODI 

DoN 

DI 

DSI 

FC 

FIP 

FIPC 

FIRMR 

FM 

FP 

FSP 

FY 

GAO 

GB 

GC 


Central  Processing  Unit 
Defense  Business  Operating  Fund 
Director  of  Defense  Information 
Defense  Data  Network 
Department  of  Defense 

Department  of  Defense  Directive  (same  as  Instruction) 

Department  of  Defense  Instruction  (same  as  Directive) 

Department  of  the  Navy 

Degree  of  Influence 

Delivered  Source  Instructions 

Unadjusted  Function  Point  Count 

Federal  Information  Processing 

Financial  Information  Processing  Center 

Federal  Information  Resources  Management  Regulation 

(published  by  U.S.  General  Services  Administration) 

Financial  Management 

Function  Point(s) 

Full-time  Software  Person 
Fiscal  Year 

General  Accounting  Office 
Gigabytes 

General  Characteristics 
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GCA 

GTS 

GSA 

GTR 

HASC 

HSETC 

IDTT 

IFPUG 

IMAPMIS 

I/O 

IR 

IRM 

IRMC 

IRR 

IT 

KDSI 

KLOC 

LCM 

LAN 


General  Characteristics  Adjustment 

Government  Travel  System 

U.S.  General  Services  Administration 

Government  Transportation  Request 

U.S.  House  of  Representatives  Committee  on  Armed 

Services  (House  Armed  Services  Committee) 

Health  Science  Education  Training  Command 

Inactive  Duty  Training  Travel 

International  Function  Point  User  Group 

Inactive  Manpower  and  Personnel  Management 

Information  System 

Input/Output 

Information  Resources 

Information  Resources  Management 

Information  Resources  Management  College  (within 

National  Defense  University) 

Individual  Ready  Reserve 

Information  Technology 

Thousands  of  Delivered  Source  Instructions 

Thousands  of  Lines  of  Code 

Life-Cycle  Management 

Local  Area  Network 
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LOC 


Lines  of  Code 


MAC 

MB 

Mbps 

MCPS 

MIPS 

MM 

MNS 

MPT 

NARDAC 

NAVPTO 

NCTS 

NIF 

NRPC 

OMB 

OPTAR 

PM 

PTSR 

PB 

PV 

R&D 


Military  Airlift  Command 

Megabytes 

Megabits  per  second 

Microcomputer  Claims  Processing  System 
Millions  of  Instructions  Per  Second 
Man-month  (of  effort) 

Mission  Needs  Statement 

Manpower  Personnel  and  Training 

Navy  Regional  Data  Automation  Center  (now  called  NCTS) 

Navy  Passenger  Transportation  Office 

Navy  Computer  and  Telecommunication  Station  (formerly 

NARDAC) 

Navy  Industrial  Fund 
Naval  Reserve  Personnel  Center 
Office  of  Management  and  Budget 
Operating  Target  (for  budget  expeditures) 

Program  Manager 
Problem  Tracking  System  Report 
Discounted  Payback  Period 
Net  Present  Value  Analysis 
Research  and  Development 


142 


RAM 


RESCOMMIS 

RISC 

RTS 

RHS 

RPN 

RSTARS 

RTSS 

SASET 

SATO 

SELRES 

SEI 

SEMA 

SIR 

SPA 

TCP/IP 

UAC 

UCA 


Random  Access  Memory 

Reserve  Command  Management  Information  Strategy 
Reduced  Instruction  Set  Computing 
Request  for  Transportation  Services 
Reserve  Headquarters  System 

Reserve  Personnel  Navy  (Congressional  appropriation) 
Reserve  Standard  Training  Administration  and  Readiness 
Support 

Reserve  Training  Support  System 
Software  Architecture,  Sizing,  and  Estimating  Tool 
Scheduled  Airline  Ticket  Office 
Selected  Reservists 

Software  Engineering  Institue,  Carnegie  Mellon  University 

Systems  Engineering  and  Management  Associates,  Inc. 

Savings/Investment  Ratio 

Software  Process  Assessment  (performed  by  SEI) 

Transmit  Control  Protocol/Intemet  Protocol 

Uniform  Annual  Cost 

Uniform  Chart  of  Accounts 
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APPENDIX  B 


PROTECT  YEAR  DISCOUNT  FACTORS 


Year  Table  A  Table  B 


PRESENT  VALUE  OF  $1  (Single 

PRESENT  VALUE  OF  $1 

Amount  used  when  cash  flows 

(Cumulative  Uniform  Series  to  be 

accrue  in  varying  amounts  each 

used  when  cash  flows  accrue  in  the 

year). 

same  amount  each  year). 

1 

0.954 

0.954 

2 

0.867 

1.821 

3 

0.788 

2.609 

4 

0.717 

3.326 

5 

0.652 

3.977 

6 

0.592 

4.570 

7 

0.538 

5.108 

8 

0.489 

5.597 

9 

0.445 

6.042 

10 

0.405 

6.447 

11 

0.368 

6.815 

12 

0.334 

7.149 

13 

0.304 

7.453 

14 

0.276 

7.729 

15 

0.251 

7.980 

16 

0.228 

8.209 

17 

0.208 

8.416 

18 

0.189 

8.605 

19 

0.172 

8.777 

20 

0.156 

8.933 

21 

0.142 

9.074 

22 

0.129 

9.203 

23 

0.117 

9.320 

24 

0.107 

9.427 

25 

0.097 

9.524 

NOTE:  Table  B  factors  represent  the  cumulative  sum  of  Table  A  factors  through  any  given  project  year. 
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